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Preface 


In this book, we summarise most of what we have learned throughout our research 
project, Governing by evaluation in higher education in Sweden. When the Swedish 
Research Council granted us resources for this project in autumn 2012, we had 
planned to study the national evaluation and quality assurance system that was in 
operation at the time. One of our major ambitions was to follow a handful of review 
processes closely from the perspective of the higher education institutions under 
review and the same processes from the national agency’s perspective. As it turned 
out, this was impossible, and we faced a rather drastic predicament for all social 
research: the social world changes, and changes may come swiftly. And they did. As 
we were about to start our project, the national evaluation and quality assurance 
(EQA) system was terminated. Instead, we faced a period marked by a lack of for- 
mal political decisions on a new national EQA system accompanied by intense 
policy work to develop and design this reformed system. 

We turned this to an advantage in that we redirected some of our research efforts 
to study topics that are seldom researched simply because they cannot be predicted 
and neatly presented in any research proposal. For example, we were able to empiri- 
cally study the interesting and important ongoing process of developing policy—in 
our case, a new national EQA system. We were also able to closely follow the 
implementation period and the pilot reviews of higher education institutions inter- 
nal quality assurance systems that were part of that process. 

The disruption in our research plan caused by this state of affairs within the 
Swedish higher education sector also became a crucial issue when we started to plan 
this book. How should we organise the chapters? In the end, we decided to combine 
a chronological structure with the overall rationale of our sub-studies; certain parts 
evolved as results of previous processes through which events unfolded and attracted 
our curiosity and interest. The book, therefore, is a historical account in which the 
emphasis is on the most recent national EQA systems in Sweden (2011-2014 EQA) 
and the present system (2016 EQA) and its development and inception. So, although 
the book is an anthology, we strived to compile a coherent account that runs through- 
out the book about contemporary work with EQA policy and practice in Sweden. 


vi Preface 


A few words also need to be said about the editorship. Five researchers have been 
engaged in the research, sharing and contributing to most of the different types of 
work that have been done: reading, interviewing, analysing, writing, administrating, 
etc. Our appreciated colleague Professor Anders Olofsson was sadly struck by a 
stroke in March 2017 and has not been able to contribute to the reanalysis of materi- 
als and writing of the texts in the book. However, he was actively involved in the 
project work, which is acknowledged in footnotes and references. He has also been 
partly engaged in our editing work, by supplying comments and suggestions for 
improvements. We want to credit his work with us and the project by recognising 
that he has done this work with us editors, as stated on the title page. 
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Governing by Evaluation: Setting A 
the Scene chee 


Christina Segerholm, Linda Rönnberg, Joakim Lindgren, and Agneta Hult 


Abstract This introductory chapter starts by outlining the aim of the book: to anal- 
yse and discuss the interplay between governing, evaluation and knowledge with an 
empirical focus on Swedish higher education. It then goes on to locate this aim and 
the intended contribution within the wider research context and in previous studies. 
The chapter also highlights some important national traits of the Swedish case and 
Swedish higher education policy development, before presenting the overall con- 
ceptual frame employed in the book and the project it builds on. Finally, an outline 
of the forthcoming chapters is provided. 


Introduction 


In this book, we address and problematise issues of how, where and why evaluation 
and quality assurance reforms are shaped, legitimised and enacted in the context of 
higher education. More specifically, the aim is to analyse and discuss the interplay 
between governing, evaluation and knowledge, with an empirical focus on Swedish 
higher education. We are interested in the pivotal role of knowledge as a governing 
resource, and we seek to highlight the particular features of evaluation as a practice 
that makes knowledge work for governing. 

We draw on extensive empirical studies and findings from the project Governing 
by evaluation in Swedish higher education 2013—2018, in which we sought to 
understand the governing-evaluation-knowledge problem, by focusing on interna- 
tional and national contextual and political frames underlying recent evaluation and 
quality assurance reforms in higher education. We also sought to understand the 


C. Segerholm (%4) - A. Hult 
Department of Education, Umea University, Umea, Sweden 
e-mail: christina.segerholm @umu.se; agneta.hult@umu.se 


L. Rönnberg - J. Lindgren 
Department of Applied Educational Science, Umea University, Umea, Sweden 
e-mail: linda.ronnberg @umu.se; joakim.lindgren@umu.se 


© The Author(s) 2019 1 
C. Segerholm et al. (eds.), The Governing-Evaluation-Knowledge Nexus, 

Evaluating Education: Normative Systems and Institutional Practices, 
https://doi.org/10.1007/978-3-030-21143-1_1 


2 C. Segerholm et al. 


enactments of these reforms in Swedish higher education institutions and in the 
responsible national evaluation agencies. While working with our project, we found 
that the term nexus captured much of our research ambition and understandings of 
these relationships. Hence, the use of nexus in the title of the book refers to both the 
meaning “connection, link” (Merriam-Webster Dictionary n.d.), and to the older, 
Latin meaning “the act of binding together” (Wiktionary n.d.), or “a binding 
together’ (Oxford Dictionaries n.d.). Throughout this book, we use the development 
of evaluation and quality assurance (EQA) systems in Swedish higher education to 
explore and analyse how governing, evaluation and knowledge are connected and 
bound together, in the activities of policy travel and brokering, decision-making, 
media coverage, design, enactment, translation and by assumptions and conceptions 
of quality. 

We recognise the intimate connection between geographical space and time and 
the need to acknowledge how the patchwork of higher education governance varies 
across (and within) nations. Paraphrasing Massey (2013), we could imagine our- 
selves taking a train across the international higher education landscape. Despite 
transnational influences and modes of governing, we would be “cutting across a 
myriad of stories going on”. Massey’s allegory identifies the higher education land- 
scape as “a pincushion of a million stories”, and Sweden offers a particular national 
framework for particular stories. Sweden was the first European country to create a 
unified mass higher education system in the 1960s and 1970s (Neave 1998). Based 
on modernist ideas of reform through social engineering (Larsson et al. 2012a), 
forms of evaluation that aimed at improving the system were developed and set up 
as a direct response to this development. From the outset, governing work depended 
on knowledge and expertise and used evaluation as a specific kind of knowledge- 
based form of enactment of governing. We might even say that epistemic gover- 
nance and Sweden is an old affaire de Coeur. So, whereas the general utopian ideas 
of using evaluation for purposes of improvement have remained intact, the welfare 
state in Sweden has undergone dramatic changes, as has the higher education sys- 
tem and the modes of governing the system by evaluation. Such forms of historical 
continuities have inspired Swedish scholars to challenge the orthodoxies of the gov- 
ernance narrative and develop more context-sensitive descriptions in terms of, for 
example, a shift from social engineering to “advanced liberal engineering” with 
emphasis on the important role of regulatory apparatuses involving “standardisa- 
tion, monitoring, auditing and evaluation” (Thorn and Larsson 2012, p. 263). At the 
same time, Sweden shares a dominant rationale for contemporary governing based 
on modernity with most countries in Europe and beyond. That is, the rationale gov- 
erning by objectives/goals and results/outcomes, which in turn requires feedback 
mechanisms like evaluation (Therborn 1995). 

The particular contemporary Swedish history, the specific continuities, displace- 
ments and breaks, makes it highly informative to zoom in on the Swedish case in 
order to explore the role of evaluation in governing higher education. As such, the 
book is a contribution to understanding governing that actively works with transna- 
tional developments and interrogates them through detailed and specific national, 
local and institutional exploration. Illustrations from a specific national case may 
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also help other researchers to identify specificities and thus to contribute to scholar- 
ship that acknowledges globalising and transnational developments but pays atten- 
tion to translation in a particular context. 


Research Context and Project Contributions 


In Europe and beyond, higher education has increasingly been targeted by political 
initiatives aimed at intensifying the societal and economic benefit from this sector 
(Shattock 2014), and researchers have been attracted to studying these develop- 
ments more closely. Common observations from a vast number of studies point to 
the expansion or massification in terms of the number of students; the changed rela- 
tions between the State and higher education; the importance of internationalisa- 
tion/globalisation; transnational governance; mergers of higher education 
institutions; a move towards market-oriented policies — including commodification 
and increased media relations; the adoption of new public management in higher 
education institutions’ (HEIs’) internal governance structures; the relation between 
academic freedom and institutional autonomy; and the shift to performance-based 
funding (e.g. Olssen and Peters 2005; Gornitzka et al. 2007a; Herbst 2009; Schuetze 
et al. 2012; Sultana 2012; Rider et al. 2013; Goodman et al. 2013; Shattock 2014; 
Cai et al. 2016; Fumasoli et al. 2017). In the European context, the significance of a 
common degree structure and of outcome-based learning objectives and standards 
(e.g. Brøgger 2018; Normand 2016) are also recurrently observed. There is indeed 
a rich literature covering different aspects of EQA in higher education in the wake 
of the developments described above. Such studies include, for instance, systemic 
and structural aspects of these relationships, issues pertaining to design selection 
and implementation of quality assurance activities and tools, as well as critical anal- 
ysis of quality assurance as regulation and occupation (c.f. Travers 2007; 
Westerheijden et al. 2007; Pratasavitskaya and Stensaker 2010; Paradeise and 
Thoenig 2013; Rosa and Amaral 2014; Enders and Westerheijden 2014; Jarvis 
2014; Leiber et al. 2015; Beerkens 2015; Brady and Bates 2016; Toots and Kalev 
2016, to give a few examples from the last decade). However, such studies have 
rarely dealt with the Swedish case. In fact, with a few exceptions, research in 
Swedish higher education (c.f Geshwind and Forsberg 2015; Wedlin 2011; Wedlin 
et al. 2017) have seldom explicitly targeted EQA practices (c.f. Gröjer 2004; 
Karlsson et al. 2014; Lindgren 2012; Kettis and Lindberg-Sand 2013). By this book, 
we seek to add to this literature, by focussing on different facets and the interplay of 
and between knowledge, evaluation and governing in higher education. 

As a point of departure, we suggest that contemporary transformations in higher 
education governance reflect moves of simultaneous deregulation, decentralisation 
and self-governing on the one hand and reregulation and centralisation on the other. 
Inherent in these activities is the dilemma of balancing (external) control/account- 
ability and support (House 1993; Karlsson Vestman 2011) in education governance 
and evaluation, as also noted in research on global and European education policy 
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(Ozga and Lingard 2006). In the words of Campano, “These complex, often contra- 
dictory governance shifts in higher education represent a process that could be of 
considerable interest, were it included in the broader debate on ‘governance’ that 
has emerged over the last 15 years in the social sciences” (2011, p. 1622). Not only 
do these transformations of higher education concern governance shifts and ten- 
sions, but they also incorporate different actors and work processes as well as the 
emergence of the so-called intermediary bodies (Neave 1998). Furthermore, the 
transformations encompass ideas of what higher education (or a university) is and 
should be (Karlsohn 2016). Here one of the key ideas in the book emerges, namely, 
that governance may transform social realities in profound ways. 

Different means are used in these political endeavours and balancing acts, and in 
this volume, we have deliberately concentrated on exploring EQA systems as part 
of these governing efforts. We find that much of the transformations noted in con- 
temporary research on higher education are in fact visible in (national/state/regional) 
EQA systems. Such systems are themselves subjects for reforms, policies and polit- 
ical decision-making, in how they best are designed, implemented and practised 
(e.g. Salter and Tapper 2000; Dang and Stensaker 2007). They may therefore cap- 
ture governing ambitions, moves and tensions, as well as the extensive policy work 
and enactments by different actors involved in these processes in diverse institu- 
tional settings. 

Governing by evaluation in higher education always presupposes and involves 
different forms of knowledge. Firstly, evaluations themselves are based on particu- 
lar epistemologies and choices regarding methodological designs, and their con- 
crete enactment involves practical forms of know-how. Moreover, such evaluations 
also necessitate some knowledge about the real world of higher education systems, 
including the formal laws and soft rules that HEIs — and evaluations for that matter — 
must adhere to. 

Fuelled by efforts manifested in the Bologna Process and the formation of the 
European Higher Education Area, organisations such as the European Association 
for Quality Assurance in Higher Education (ENQA) and their Standards and 
Guidelines for Quality Assurance (ESG) (2015) are increasingly influential in gov- 
erning higher education across nations and systems (Enders and Westerheijden 
2014). These policy developments also carry potential (re)locations of power and 
relationships, in which national systems and higher education institutions are navi- 
gating their roles and functions (Lingard and Rawolle 2011). 

We argue that Sweden is a particularly productive site for examining the com- 
plexities of governing higher education in the context of EQA. Several national 
reforms have been implemented over the last three decades (see the section on the 
Swedish case), producing certain governing tensions, reflecting moves of the simul- 
taneous deregulation and decentralisation of self-governing on the one hand and 
reregulation on the other, manifested in: 


(i) Problems of balancing control/accountability and development/enhancement 
(learning). 
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(ii) Challenging international-national-local institutional relationships, for 
instance, in the exclusion of the Swedish national agency as a member of the 
strong policy organisation the ENQA. 

(iii) An almost 2-year-long period in which no reform decision of a new national 
EQA system was taken, but tensions were intended to be restrained. 


These particular circumstances formed a unique opportunity to study the con- 
temporary relations between governing, evaluation and knowledge in higher educa- 
tion and its local institutional, national and international (European) policies and 
practices. In taking this approach, our research reported in this book: 


(a) Provides insights into the power of European policy flows and activities con- 
cerning EQA in higher education and their national and institutional enact- 
ments. We see that the interrelatedness of European and national policy 
activities, paired with a specific national reform situation produced a powerful 
governing context, in which higher education institutions have to act on and 
reconcile various internal and external demands and conceptions of quality in 
higher education. 

(b) Contributes understandings of governing by evaluation at a point in time when 
one national EQA system was terminated, but the content and design of the new 
system that was to replace it was pending for almost 2 years. Previously, the 
systems followed one another without interludes, and this temporary halt was a 
new situation for the higher education sector. By studying national politics and 
policy processes in this “interval” between two major quality assurance reforms 
and by analysing how higher education institutions responded to and handled 
national and international policy signals, we show how governing is “done” 
when a national reform is expected but not yet decided. 

(c) Generates findings from a particular research design (see the Appendix) in 
which the evaluators and the evaluated were studied in parallel, facilitating a 
holistic view of external evaluation processes. This design gave us knowledge 
of how certain governing signals in the evaluation processes are enacted in the 
work of higher education institutions. As will be shown, this enactment varies 
and is infused by local institutional contexts, knowledges and experiences. 

(d) Provides knowledge from the rare opportunity to study national EQA reform 
“in the making” and its short-term influence on education policy and practice in 
real time. The design allowed us to study how the policy for a new national 
EQA system is made and put into practice in the Swedish higher education 
system as the events unfolded. This gave us knowledge of, for instance, the 
stress put on European policy, the importance of actors’ conceptions of quality 
in higher education, of what types of knowledge are required to translate policy 
into practice, to examine, measure and assess quality, and hence make knowl- 
edge work for governing in evaluation processes. 


Furthermore, the book offers opportunities for comparisons to Sweden for 
nations that may share some of the particular characteristics of the Swedish case, in 
order to gain collective insights, understandings and knowledge about the 
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contemporary governing of higher education and how evaluation, quality assurance 
and knowledge are inextricable parts of this. The book contributes to descriptions 
and analysis of how evaluation expands over time and reaches out and involves an 
increasing number of actors and activities at different levels, nationally and interna- 
tionally, in the governing of higher education. Therefore, the book also responds to 
the ambition expressed by Furubo and Stame (2019): to provide “‘a critical view” of 
evaluation, scrutinising it within a wider societal context, something not done that 
often. As called for by scholars from different research perspectives, the book offers 
an illumination of the relationship between the construction of policy and practice 
in higher education (Wedlin 2011, pp. 46—47) or what Gornitzka et al. (2007b, 
p. 13) identified as “[a] need to find ways of looking at the relationship between 
policies and practices in higher education”. From our rich empirical material, we 
also describe in some detail “the missing link”, that is, what happens with political 
signals and decisions in the everyday practice of higher education. We draw on our 
empirical work to illustrate and problematise different aspects and facets of the rela- 
tions between governing, evaluation and knowledge in the context of Swedish 
higher education. 

In the remainder of this chapter, we describe the case of Sweden and some gen- 
eral features of higher education governance. We also note the significance of the 
Bologna Process and the subsequent policy and governing work in the reshaping of 
the higher education system during the last decades. We then discuss the conceptual 
triad that is located at the core of our research interest, namely, governing, knowl- 
edge and evaluation. The chapter ends by introducing the forthcoming chapters. 


The Swedish Case 


Although rather large in geographical terms, Sweden, with its approximately 10.2 
million inhabitants, is not a big country in population. During a major part of the 
1900s, it was a rather homogenous country in that it had a protestant state church, 
Swedish as administrative and major language, and was ruled for more than four 
decades by the Social Democratic party (1932—1976). Politically, Sweden is known 
for the Social Democratic party’s control of the economy by central planning, along 
with corporatist arrangements (c.f. Rothstein 1992). The famous so-called 
Saltsjöbaden spirit (Larsson et al. 2012b, p. 16-17) came out of an agreement in 
1938 between the workers’ union and the employers’ organisation. The agreement 
stipulated that the labour market should not be subject to regulations by the govern- 
ment. Instead, labour market issues should be handled by the different labour mar- 
ket actors in consensus-seeking processes without interference of the government. 
Another signifying trait of the Swedish post-war state was of course its social wel- 
fare expansion and growth. Swedish welfare came to be characterised by general 
distribution of social welfare and high public spending in areas such as social ser- 
vices and insurance, health care and education. This resulted in a particular form of 
social democratic welfare state (Esping-Andersen 1990) in which comprehensive 


Governing by Evaluation: Setting the Scene 7 


and publicly financed welfare were conceived as the norm. Taken together, institu- 
tional arrangements like these are argued to have built trust in the state and also gave 
some stability for the welfare society project, aided by a culture of consensus- 
seeking procedures and extensive consultations prior to reforms (Larsson et al. 
2012b, p. 16-17). 

From the 1930s and onwards, the development of the welfare state through cen- 
trally planned piecemeal social engineering took off (Larsson et al. 2012b, p. 16-17). 
Trust and social engineering are important characteristics for Sweden, and political 
decision-making and extensive reforms were to be based on expert knowledge 
(Larsson et al. 2012b, p. 12-14). Such expert knowledge was achieved through the 
recruitment of experts to national committees representing different interests in 
society. Reports from the committees were sent out to various stakeholders for con- 
sultation before political decisions were taken. This government rationale was in 
place more or less up until the 1980s, when the economic crisis of the 1970s put an 
end to the expanding welfare state with its social and educational reforms. 

In general, Swedish politics, with its proportional electoral system and multi- 
party system, has often been seen as a culture of relative political consensus, along 
with willingness to cooperate and negotiate. A history of minority governments and 
the need for bargaining and coalitions have counteracted more confrontational “the 
winner takes it all” political approaches, as promoted, for instance, by the English 
political system (c.f. Lewin 2002). However, the last decades witnessed an increas- 
ing political polarisation in the form of two dominant blocs (a predominantly social 
democratic and centre-right, respectively). The most recent elections have compli- 
cated this picture, as a growing populist right-wing and anti-immigrant party is 
holding the balance of power between the blocs (Aylott and Bolin 2015). The 2019 
minority government coalition (Social Democratic and Green Party) came into 
office after seeking support from some of the parties in the former centre-right bloc. 

Looking closer at education policy, a signifying trait of the post-war era has been 
the political advocacy of equivalence (sometimes translated to equity) as a means to 
promote equality, increasing social justice and mobility and to counteract the effects 
of an uneven distribution of resources. However, economic and societal transforma- 
tions of the Swedish welfare state have contributed to gradually transform education 
to be conceived as a “private good” rather than a “public good” (Englund 1993). A 
focus on individual freedom of choice, individualised responsibility, competition 
and individual capacity building now serve as general guiding principles for educa- 
tion policy (Englund 1993; Dahlstedt and Fejes 2019). The social justice dimension 
once incorporated in the notion of equivalence has been challenged and arguably 
infused with new meanings. It should however be noted that higher education in 
Sweden is free of charge to Swedish and European students, and there is still a 
strong policy emphasis on widening access and participation in higher education, 
along with strivings to actively recruit underrepresented student groups (c.f. 
Government Offices of Sweden 2017). 

To sum up, the strong social democratic heritage and power as a form of “hege- 
monic force” (Agius 2007, p. 585) has framed the history of the Swedish welfare 
state. Although influenced by several characteristics of New Public Management 
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(Pollitt 1995), neoliberalism has not become what Braithwaite (2005, p. 3) calls “an 
institutional reality” in Sweden, as neoliberalism has not been institutionally effec- 
tuated in terms of “a diminished public sphere”. The influx of market ideas has 
increased the role of regulation in shaping policy and politics, which is highly evi- 
dent within the education area. Market ideas and consumerism have increased the 
need for bureaucratic regulation, audit, inspection and other forms of control. We 
find it therefore apt to lean to ideas of “regulatory capitalism” as a broad description 
of the contemporary regulatory design and institution — “one that is being consti- 
tuted, shaped, constrained and expanded as a historically woven patchwork of regu- 
latory institutions, strategies and functions” (Levi-Faur 2017, p. 289; see also 
Braithwaite 2005). In the next section, we move from general historical and social 
depictions to the particular characteristics of Swedish higher education. 


Swedish Higher Education and Its Policy Context 


The majority of the 48 Swedish HEIs are public. There are also a small number of 
independent higher education providers that predominantly give courses within a 
specific and more limited subject area. Today, about 350,000 students attend these 
HEIs. The number of students enrolled in Swedish higher education has increased 
considerably over time. Going back to the 1977 higher education reform, when 
most postsecondary education was organisationally relocated to higher education, 
the number of students increased to reach about 160,000 in the early 1980s. An even 
more intensified student expansion was evident in the 1990s and the 2000s, to reach 
a high point with 365,000 students in 2010 (the Swedish Higher Education Authority, 
SHEA 2018). Higher education staff are employed by the state, making higher edu- 
cation the largest public sector in terms of the number of persons employed. In 2017 
there were more than 75,000 HEI employees (the corresponding figure for full-time 
equivalents is about 60,000). Research and teaching staff make up about 60 per cent 
of all employed in Swedish higher education. The expansion of higher education is 
discernible in the number of research and teaching staff as well, which has increased 
over time. This also goes for the number of hired administrative personnel in HEIs. 
This category of employees has increased by almost one third in the last decade 
(SHEA 2018). 

Within a government context, it is important to recognise that Sweden has a 
political system in which the ministries within the government are small compared 
to many other countries. The national agencies are part of the state in that they (ide- 
ally) implement and administrate parliamentary and government decisions, and 
they are separated from the government office and the ministry offices. Swedish 
agencies are often portrayed as largely autonomous compared to many other coun- 
tries. The autonomy, independence and accountability of agencies are regulated by 
laws adopted by the parliament and by ordinances and provisions issued by the 
government. Each agency is also governed by annual appropriation directions, 
which regulate the activities, objectives and economical resources for the agency. 
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This arrangement means that a ministry and its political ministers are not allowed to 
intervene in individual matters handled by the agency (known as ministerial rule). 
Administrative discretion, coupled with extensive informal contacts with the minis- 
try, along with agency employees’ expertise, place the agencies in an influential 
position (Rothstein 2005; Pierre 2004). 

In the case of higher education, there are two major national agencies: the 
Swedish Higher Education Authority (SHEA, Universitetskanslerscimbetet, UKA) 
and the Swedish Council for Higher Education (Universitets- och hégskoleradet, 
UHR). The former is responsible for national supervision, EQA and some other 
government assignments, and in this book this intermediary body will play a central 
role. The latter agency works with admissions to higher education, evaluates foreign 
students’ qualifications and brokers international exchange, and thus its scope is not 
within the primary interest of this book. 

The Higher Education Act (SFS 1992:1434 n.d.) and the Higher Education 
Ordinance (SFS 1993:100 n.d.) apply for all public HEIs, with some special regula- 
tions for the Swedish University of Agricultural Sciences and the Swedish Defence 
University. They also receive annual appropriation directions from the government 
concerning the use of the annual budget and how to report back to the government. 
For some HEIs, this means that they are commissioned to undertake special assign- 
ments, for example, decentralised medical education or education in minority 
languages. 

The Swedish higher education system mainly consists of two kinds of HEIs: 
universities and university colleges. Universities have a more far-reaching right to 
award degrees compared to university colleges. University colleges have to apply to 
the national agency for the right to award degrees and certificates for degrees in art, 
professional degrees, and master’s and doctorate degrees, while universities have to 
apply for the first two types of degrees and are free to award master’s and PhD 
degrees. Independent providers receive degree-awarding powers from the govern- 
ment. However, as we show in the chapter “National Evaluation Systems”, these are 
conditional rights that may be revoked, should the HEIs not live up to the require- 
ments set in national EQA exercises. Universities are often larger than university 
colleges, meaning that they comprise of several faculties and scientific areas. Some 
HEIs are specialised in, for instance, technology, medicine or art. 

Higher education is free of charge for citizens within the European Union, the 
European Economic Area and Switzerland. Before 2011, higher education was free 
of charge for all. Higher education in Sweden is funded by tax revenues. Some other 
arrangements also exist but are scarce. From the 1993 reform (Government Bill 
1992/93:1), performance funding was introduced, meaning that a per capita revenue 
is allocated for each student registered on a course, and a per capita revenue is allo- 
cated for each student who fulfils the course requirements. This reform also intro- 
duced mandatory internal quality assurance at the HEIs and could probably be 
described as the first move after the Second World War towards market orientation, 
since it introduced incentives for the HEIs to compete in order to attract students. 
This reform was also a move towards increased institutional autonomy, in that the 
HEIs could now decide on curricula (with the exception of professional programmes) 
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that earlier were centrally decided. In the comprehensive study by Bauer et al., they 
summarised the shifts as follows: 


The shift in authority was primarily characterized by the move away from centrally regu- 
lated and steered institutions to more autonomous institutions, led by more powerful insti- 
tutional leaders who were now to compete in an education marketplace. Such a shift in the 
distribution of authority between the state and the institutions naturally brought about a 
change in the authority in the central bureaucracy, with a new emphasis on accountability 
rather than on planning and managing the system. (Bauer et al. 1999, p. 101) 


The entire degree structure of the Swedish system changed in 2007, when the 
“three-cycle” system (degrees at three levels) from the Bologna agreement was 
decided and Sweden abandoned the previous structure with two levels (Government 
Bill 2004/05:162; Parliament Standing Committee on Education 2005/06:UbU3). 
The influence of European policy became apparent with this reform and clearly 
spelled out in the government’s motivation of the proposed change: 


In order to contribute to Swedish higher education’s international comparability, attraction 
and currency, the Government proposes and make assessments concerning changes in the 
educational and degree structure in Swedish higher education. This is done particularly on 
the basis of Sweden’s participation in the so-called Bologna process, which today includes 
over 40 European countries. This process aims to promote mobility, employability and the 
competitive power of Europe as a continent of education. (Parliament Standing Committee 
on Education 2005/06:UbU3 n.d., p. 1) 


This parliamentary decision led to a revision of all national requirements for 
degrees at all three levels. It also led to a major revision of local plans for different 
education programmes and subject courses at the HEIs. Programmes and courses 
should, from then on, be based on the rationale of the relation between expected 
learning outcomes (learning objectives) and acquired learning outcomes and 
whether or not the latter was in line with the requirements for a specific degree. 

These developments have continued with a reform to further strengthen HEI 
autonomy (Government Bill 2009/10:149), leaving the HEIs to decide on internal 
organisational structure, on types of positions and requirements for employment, 
and to allocate resources internally at their own discretion. From the 1993 reform 
onwards, a number of national EQA systems have been decided, implemented and 
operated, which we will analyse in more detail in the chapter “National Evaluation 
Systems”. Next, we will elaborate on our general understanding of the three con- 
cepts that we see as central to explore the nexus. 


Understanding Governing, Knowledge and Evaluation 


As an overall point of departure, we have used governing, knowledge and evalua- 
tion as a conceptual frame. The different chapters also draw on additional theoreti- 
cal resources, some more and others less, in order to bring out relevant perspectives 
from the presented data. We have adopted an eclectic approach to further our under- 
standing of these processes, and recognising this deliberate theoretical plurality has 
been an important basis for our joint work in the research project. 
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Governing 


Our interest in researching governing goes back to our previous projects on quality 
assurance, school inspection and education governing in compulsory education (c.f. 
Ozga et al. 2011; Grek and Lindgren 2015; Carlbaum et al. 2014). Here, governing 
is conceptualised as a verb, as a way to emphasise the actual work and doings of, 
and in the policies that governing entails. This formed our overall conceptual 
approach to governing, and we draw on this to research governing of higher educa- 
tion and the work that are done in these processes. Our approach to governing draws 
attention to the work of actors and their mediation. They engage in activities that 
build on and foster certain knowledge that move across nations and contexts in par- 
ticular ways, which relies on data and comparison as sources of legitimacy. 

We understand governing as the activities composed of assemblages of places, 
people, policies, practices and power (Clarke 2015, p. 21). This notion of governing 
was put forward as a way of trying to “focus on the complexity, contestation and 
translation of governing practices that avoided the system-theory references of gov- 
ernance and the totalising tendencies of governmentality” (Clarke 2015, p. 12-13). 
This conceptualisation, however, does not entail a view of the complete hollowing 
out of the state (c.f. Rhodes 2007) but rather points to the practices and doings in 
diverse, complex and multilevel arrangements and acknowledges that (state) gov- 
erning in itself is a way to make this diversity of arrangements governable (c.f. 
Pierre and Peters 2000). However, the state is not a monolithic entity but encom- 
passes a multitude of actors, opinions and actions, as well as the tensions within and 
between them. In this context, neo-institutional approaches and insights from organ- 
isational studies are useful in order to further analyse such governing complexities 
(c.f. Jacobsson et al. 2015). 

Furthermore, ideas on decentred governance draw attention to the importance of 
actors’ contingent desires, beliefs, preferences and intentions as part of meaningful 
actions and activities of governing, that is, that governing works through processes 
where actors “create and act on meanings” (Bevir 2011, 2013, p. 56). This means 
that we pay careful attention to individuals by examining “the ways in which pat- 
terns of rule, including institutions and policies, are created, sustained, and modified 
by individuals whose actions (...) arise from the beliefs individuals adopt against 
the background of traditions and in response to dilemmas” (Bevir, 2013 p. 65). 

We see the acts of governing as a set of multiple processes, involving different 
forms of work, for instance, through mobilisation of agents and agencies in order to 
realise certain aims. National EQA systems may be used and adapted to suit very 
different circumstances and expectations. EQA systems respond to and are affected 
by changing political demands and display both gradual and more drastic processes 
of change. The stress on governing as work and doings has led us to recognise the 
importance of policy enactment (rather than implementation) (Ball et al. 2012). 
Enactment is used to draw attention to that policy, as encoded text must be decoded 
in concrete environments, often in messy and non-linear processes of interpretation 
and translation. As Ball and his colleagues argue: “Enactments are always more 
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than just implementation, they bring together contextual, historic and psychosocial 
dynamics into a relation with texts and imperatives to produce action and activities 
that are policy” (Ball et al. 2012, p. 71). With our approach, we want to highlight 
that the work of “doing EQA” entails enactments that are embedded in different 
institutional and organisational contexts and that local contexts matter. We are also 
interested in the political work that is being done, the speed and pace at which these 
governing activities are designed and enacted (c.f. Jessop 2015), as well as in the 
actors that broker and/or carry knowledge and “do” policy. In this book, we will 
introduce the reader to a particular group of people we have named “qualocrats” 
who carry and embody such important knowledge and expertise in the field of EQA 
in higher education. 

We claim that “doing” governing is dependent on sharing and negotiating knowl- 
edge, to produce (certain) knowledge and to define what counts as valid knowledge, 
as well as creating solid bases for decisions in the evaluation of central issues. Of 
particular interest to us is “meditative” governing (Jacobsson 2010), which points to 
the importance of compared experiences and shared ideas in doing governing. The 
concept is related to the idea of policy learning and teaching, translation, brokering 
and networking, as in the “actual work” (Sassen 2007, p. 37) of governing. As Ozga 
(2016, p. 71) states, we can also discern the emergence of 


new kinds of governing work from particular groups of actors who are positioned at key 
points of intersection of knowledge production and practical problem-solving. This work 
demands skills in translating information into ‘practical knowledge’, mediating conflict and 
brokering interests. (Ozga 2016, p. 71). 


In our project and in the forthcoming chapters, the EQA governing work of such 
actors and the knowledge brokered in these translations and interactions are explored 
further. 


Knowledge 


As a knowledge problematic, governing by evaluation in higher education can 
essentially be understood as a state practice involving two steps. First of all, the 
state needs knowledge about the higher education sector. Borrowing an analogy 
from Scott (1998, p. 2), we might say that the Swedish state, as it struggled to get a 
handle on the expanding higher education system in the 1960s, was “partially blind; 
it knew precious little” about the inner workings of higher education institutions, 
whose creation and transmission of knowledge were underpinned by a principle of 
university autonomy, which allowed them to organise their work without being par- 
ticularly restricted by, or accountable to, outside bodies. Back then, programmes 
were designed, delivered and assessed, and standards were defined and set, within 
small, local, homogeneous and well-socialised academic communities. These 
appeared, in the eyes of the state, as obscure as a wild forest with its rich flora of 
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minerals, insects, animals, grasses, flowers, mosses, shrubs and trees, with their 
mycelial networks, mycorrhizal fungi and pheromones. 

In the next step, as carefully documented by Scott (1998), the state’s solution to 
the problems of representing complex and illegible local practices is often to remake 
them. The Swedish state used rationalist and centralist measures like central plan- 
ning, resource allocation and detailed national regulations concerning study plans 
(Gröjer 2004; Askling 2012). As noted earlier in this introduction, new modes of 
governance have replaced these efforts. We might think of the contemporary remod- 
elled higher education landscape, based on the credit-based modular formats, as an 
equivalent to the mono-cropped scientific forest that appears easy to survey, mea- 
sure and evaluate. 

Regardless of the forms of state governing, evaluation has been a prime tool used 
in order to retrieve knowledge. But evaluation not only has transformative effects on 
social realities. Social realities can also be transformed in order to be legible to 
evaluation — in order to be seen by the state (Scott 1998). Once again, we touch on 
the important issue of transformation and, more specifically, on questions about 
forms of practices, values, knowledge and potential that might be lost in such 
transformations. 

As noted above, governing by evaluation — the work of formulating and enact- 
ing — also involves different forms or phases of knowledge. In order to explore the 
role of knowledge, we draw on the work of Freeman and Sturdy (2015), who con- 
ceptualise knowledge as embodied, for example, through tactic and verbal experi- 
ence, inscribed, in different forms of texts and artefacts, and enacted, via what is 
actually done or carried out. Using this three-phase conceptualisation, we identify 
and describe forms and movements of knowledge that are manifested, incorporated 
and transformed in governing by evaluation as a social practice encompassing sev- 
eral arenas and groups of actors. 

Starting with embodied knowledge, it refers to “the knowledge held by human 
actors and employed and expressed by them as they go about their activities in the 
world” (Freeman and Sturdy 2015, p. 8). In this book, we will show that the exper- 
tise of key actors, such as the above-mentioned qualocrats, is absolutely vital. Such 
knowledge includes specific knowledge of higher education governance, institu- 
tional and organisational design and experiences from EQA. In times characterised 
by rapid change and speed, the plasticity and flexibility of such knowledge become 
particularly important. 

The upcoming chapters will also show how documents in the form of policies, 
guidelines, reports, self-evaluations and so on are equally imperative to evaluative 
activities. Such inscribed knowledge serves to model, inform, standardise and coor- 
dinate actors’ work and entail “particular ways of seeing, thinking and knowing; 
such artefacts can consequently serve to constrain and discipline our interactions 
with the world and with one another” (Freeman and Sturdy 2015, p. 11). Inscribed 
knowledge can be stored and travels in time and space. Importantly, inscription is 
often, at least temporarily, an end product that obfuscates the material process that 
gave rise to inscription (Latour and Woolgar 1979). We have sought to acknowledge, 
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or “unbox”, these material processes and the work of interpretation and translation 
that are ineluctable aspects of working with inscribed knowledge. One example is 
the production of text for different audiences. As we will show, writing and editing 
text in the context of national EQA often require collective efforts. This leads us to 
the third phase of knowledge, enacted knowledge, which is the form embodied and 
inscribed knowledge takes when expressed in doings and actions, for instance, the 
ways in which new knowledge is generated as people meet, use and share embodied 
and inscribed knowledge. Although embodied and inscribed knowledge inform and 
frame actions, enacted knowledge is never totally determined. Thus, we find that 
this concept is particularly productive when studying the concrete knowledge use 
and production of policy actors in evaluative and governing activities that cherish 
certain bureaucratic ideals, for instance, in terms of formal justice and comparative 
consistency (Molander 2016). As noted by Freeman and Sturdy (2015, p. 15), 
enacted knowledge is “characterized by a high level of interpretative flexibility 
which means that one instance of enactment may differ very significantly from 
another, even when both instances draw on the same embodied and inscribed knowl- 
edge”. In this book, we use this conceptual scheme as a way to explore what forms 
of knowledge are in operation in the work of governing by evaluation. In our 
research project, we have focussed on how different forms of knowledge move, take 
shape and are reshaped in the course of evaluation reform and activity, as actors go 
about in their work to make things happen (or not) in the context of these 
processes. 

In order to “make things happen”, evaluation reform has also drawn on knowl- 
edge of human conduct developed in the social sciences during the twentieth cen- 
tury. Contemporary ideas of total quality management, which are permeating 
mandatory internal quality assurance systems in higher education, were developed 
from scientific management via the human relations movement, with increasing 
emphasis on humanisation through self-government, empowerment, involvement 
and consensus (Boje and Winsor 1993; Barrow 1999; Behrent 2013). In this book, 
we notice how knowledge that reformed modern industrial organisations in the 
1940s is embraced by contemporary higher education reform. Such parallels open 
up a range of critical questions, in terms of the effects of particular forms of govern- 
ing through evaluation, on actors’ subjectivities and their work and knowledge. 


Evaluation 


We see evaluation and quality assurance as closely related social processes. In our 
case, both involve making judgements and producing descriptions about higher 
education, and we therefore do not separate the two in a conceptual sense. It is also 
important to underline that these processes are, by definition, about assessment and 
judgement, and therefore heavily laden with values (House and Howe 1999). 
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As noted above, our theoretical approach to governing acknowledges the work 
and “doings” of policy actors, as well as the importance of aims and directions for 
future achievements and/or projected future states. The basic rationale of evaluation 
encompasses assessments about the condition or worth of something (such as higher 
education), in order to provide an outline of how to move forward, hence the link to 
governing. Or as Mark and Henry put it: “the link between evaluation and the bet- 
terment of social conditions is absolutely crucial as a collective raison d’étre of 
evaluation” (2004, p. 36, italics in original). This ambition can be organised and 
performed in several different ways, for example, by different types of evaluations 
and/or quality assurance activities/systems. 

In this book, we will show that different EQA policies and practices rest on vari- 
ous epistemologies (Dahler-Larsen 2012a), meaning that they are designed and car- 
ried out in different ways and build on already existing knowledge. However, EQA 
also generates knowledge for formative purposes, be it development, improvement, 
control or accountability — all of them oriented to the future in a “rational” way. 
Furthermore, EQA entails actors, agencies, policies, places and work, in order to 
carry out the processes necessary, something we portray throughout the book. In 
these processes and activities, diverse forms of knowledge are in use, and produced, 
as already pointed out. In the processes of governing through knowledge, evaluation 
has also come to be understood as enclosing technologies such as visibility, compa- 
rability, sanctions and rewards, which have a productive capacity to shape behav- 
iour, elicit action and even create new ways of being a HEI actor. Thus, through the 
work, and the knowledge that is activated, EQA also does something to what is 
evaluated, reviewed or assessed, and that can be understood as part of the governing. 
Dahler-Larsen (2012a, b) perceives this influence as “constitutive effects” and dis- 
cusses how indicators are central to such influence (Dahler-Larsen 2012b, p. 173). 

In EQA, indicators and/or standards play a significant role in directing attention, 
raising expectations and pointing out what is considered valuable, important and 
desirable and what quality (in higher education) consists of. Through indicators and 
standards, standardisation takes place and makes comparisons possible, something 
that is central for competition and choice in a market-oriented higher education 
system. At the same time, other issues become ignored. Dahler-Larsen (2014) 
claims that indicators/standards therefore represent and enhance particular views of 
education quality, defining interpretative frames and world views, content, time 
frames, social relations and identities, and change their meanings as a result of their 
use (Dahler-Larsen 2014). We will point to some of these potential constitutive 
effects that the Swedish EQA systems activate. 

Over time, evaluation has evolved as a societal phenomenon and practice. From 
being a one-at-a-time rather delineated process, like programme evaluation, it has 
successively expanded in scope and comprehensiveness, over capacity building in 
organisations (e.g. Hueftle Stockhill et al. 2002; Preskill and Boyle 2008) to perma- 
nent systems with several interlinked evaluative activities (e.g. Segerholm 2006; 
Leeuw and Furubo 2008). Dahler-Larsen (2012a) likens this expansion to evalua- 
tion machines, claiming, among other things, that evaluation machines “embody a 
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set of cultural values emphasising risk management, quantification, standards, and 
a pre-emptive or prospective approach to quality control” (2012a, p. 182). He con- 
tinues to say that they “are comprehensive and general in coverage” and that “Reality 
must now become generally evaluable and thereby fit the demands of the evaluation 
machine” (ibid.). Finally, he argues that evaluation machines “seek to substitute 
subjective judgement for some objective evaluation based on standards, manuals, 
handbooks, procedures, or indicators” (ibid.). We will make use of this evaluation 
machine analogy in our explorations of evaluation as a practice of governing the 
Swedish higher education case and use the notion “evaluation machinery” to denote 
the assemblage of elements that these explorations identify. 


Outline of the Book 


In the forthcoming nine chapters, the reader will discern a movement from the more 
general to the rather detailed, as we shift our attention from broader historical and 
international developments to more thorough empirical accounts of contemporary 
affairs. The later chapters deliberately provide rather comprehensive accounts of 
evaluative systems and activities from our empirical data. We have intentionally 
strived to provide cautious and extensive empirical documentation, arguing that this 
holds an empirical value, in times of rapid restructuring and change. This book 
therefore moves from the general to the more detailed and attempts to cover differ- 
ent actors, organisations, analytical approaches and levels that, taken together and 
combined, provide a holistic account of EQA in Swedish higher education. 

In the chapter “National Evaluation Systems” we set the national scene for our 
account about the recent policy and governing processes related to EQA in Swedish 
higher education. This is done through a historical perspective, in which we describe 
and analyse national EQA systems, their overall designs and some of their conse- 
quences from the 1990s to the 2011-2014 system. In the 1960s, national evaluations 
of higher education began to be conducted, and successively, national systems were 
developed. Through the different EQA systems, certain governing signals have been 
conveyed, making the HEIs used to external evaluation and to expect constant 
changes as new systems have been implemented. Systems changed, but with some 
variation over the period, and incorporated an increasing number of activities, peo- 
ple and higher education institutions, academic subjects and programmes. We dis- 
cuss this in terms of an emerging and expanding “evaluation machinery” in Swedish 
higher education. 

However, the EQA processes we are interested in are not merely restricted to the 
national but have extended to an international and European domain. In the 
third chapter “Europe in Sweden”, our account of EQA in Swedish higher education 
therefore continues by extending the exploration to include the influx of European 
education and quality assurance policy to national policy in Sweden. Here, the 
ENQA’s membership requirements played a significant role in the Swedish policy 
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debate, which also illustrates Sweden’s embeddedness in a wider policy context. In 
this chapter, we also describe and analyse the channels by which European policies 
are disseminated to and within Sweden, pointing to the different types of activities 
such dissemination processes involve. We discuss the role of the state in this EQA 
policy transfer and conclude that the number of different types of actors, both indi- 
viduals and organisations, indicate that authority concerning higher education is, at 
the same time, dispersed and concentrated. 

Chapter four “Navigating Higher Education Institutions in Times of Quality 
Assurance: The Assumptive Worlds of Vice Chancellors” moves from the interna- 
tional and European scene to the Swedish HEIs and the mindsets of an influential 
group of actors, the vice chancellors. As responsible for the quality assurance work 
of their HEIs, our interest was directed at their ideas of quality in higher education 
and of the idea of a university, as well as of the national EQA. Their ideas are 
described and analysed as parts of their “assumptive worlds” (Marshall, Mitchell 
and Wirt 1985), understood as a common ground constituting them as a potentially 
strong influencing force in these matters. The assumptive worlds of the vice chan- 
cellors included conflicting ideals, where old traditional ideas coexisted with newer 
ones adapted to meet global economic demands. In spite of this, they were able to 
join forces and act on their critique of the 2011-2014 national EQA system. 

In the next chapter “Hayek and the Red Tape: The Politics of Evaluation and 
Quality Assurance Reform: From Shortcut Governing to Policy Rerouting”, we 
draw attention to the important political dimension of the governing-evaluation- 
knowledge nexus. We analyse the two most recent national EQA systems and show 
that the (re)construction of an evaluation machinery is far from neutral and uncon- 
tested process. We discuss how EQA systems are framed by certain ideological 
beliefs, manifested in their design as well as in the processes leading up to their 
design. We also highlight the style and speed of quality assurance policy develop- 
ment and point to the ways in which dialogue and consensus building in the 2016 
EQA reform is positioned as countering the “shortcut” policy style characterising 
the development of the contested 2011-2014 EQA system. 

The highly debated 2011-2014 national EQA system is further analysed in 
the chapter “Quality Evaluations and the Media” but approached from a different 
angle: the chapter analyses the intersection of high-stakes national higher education 
evaluations, media communication and PR strategies from the responsible agencies 
and HEIs. The study shows that the intense debate on the legitimacy of the 2011- 
2014 EQA system, during its implementation, was largely absent in the analysed 
media display of individual HEI evaluation results, as well as in the attempted fram- 
ings and bureaucratic branding activities undertaken by the responsible agencies 
and the HEIs themselves. These results suggest that once the evaluation machinery 
is in operation, it becomes hard to criticise, and the formats of media communica- 
tion, paired with logics of comparability and competition, may hamper critical 
debate. 

In chapter seven “Enacting a National Reform Interval in Times of Uncertainty: 
Evaluation Gluttony Among the Willing”, we turn to an exploration of reform activ- 
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ities within four HEIs. This chapter offers an empirical illustration of the governing- 
evaluation-knowledge nexus by pinpointing a particular situation: the time period 
between two national EQA reforms, when a new national EQA system was being 
planned and prepared, but its final design was not yet decided. We show how uncer- 
tainty during this “reform interval” opens up a potential space for policy-makers 
and HEIs to navigate. The results demonstrate that tendencies of homogenisation 
and isomorphism are strong among the four HEIs. Apparent is also the ongoing 
trend in terms of expansion, as the HEIs willingly dedicate resources to develop and 
engineer their internal quality assurance systems. 

In the following two chapters, we scrutinise the relaunching and preparation of a 
national EQA system after the reform interval and the enactment of this 2016 sys- 
tem in the form of a pilot prior to its full-scale implementation at two HEIs. In 
the chapter “Re-launching National Evaluation and Quality Assurance: Expectations 
and Preparations”, we describe and analyse the design work of this system as pro- 
cesses of governing. This work resulted in a very comprehensive system, with sev- 
eral different types of evaluations directed at most parts of the HEIs. Through the 
reintroduction of institutional reviews of the HEIs’ own internal quality assurance 
systems, the new design was in itself an expansion compared to the previous sys- 
tem. Furthermore, the European EQA policy became firmly integrated in the design 
work. We discuss how this design emphasises the governing by objectives and out- 
comes logic, promote a certain notion of quality in higher education and suggest 
that this design opens up for potential constitutive effects. 

Chapter nine “Re-launching National Evaluation and Quality Assurance: 
Governing by Piloting” explores and discusses enactments in the process of piloting 
institutional reviews. This chapter draws attention to two empirical cases and par- 
ticularly the work and experiences of different actors within the SHEA, assessment 
panels and the HEIs under review. We demonstrate, in some detail, the amount and 
forms of work done in these processes, and the important role of what we have 
termed “qualocrats” in operating the evaluation machinery. Their embodied form of 
expertise is mobilised as they move between and across different domains, to enact 
and promote certain knowledge in and of EQA. The chapter goes on to suggest that 
the pilot opened up for mutual adjustments, learning and dialogue that worked in 
order to smoothen the subsequent broader implementation of the institutional 
reviews. At the same time, it also gave rise to uncertainty and contradictory antici- 
pating governing signals. 

In the final chapter “Evaluation Machinery, Qualocrats, and the Seemingly 
Inevitable Problem of Expansion”, we highlight some of our observations on EQA 
policy and practice in Swedish higher education. We revisit the notion of an increas- 
ingly institutionalised evaluation machinery and discuss the role of qualocrats and 
judgements along with the expansion and increasing complexity of EQA work in 
higher education. 
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Abstract The national reform context for evaluation and quality assurance (EQA) 
in Swedish higher education along with the designs of the different national EQA 
systems during 1995-2014 is described and analysed in this chapter. This historical 
account establishes the background for the coming chapters in the book. Our aim is 
to scrutinise the relationship between the EQA systems and governing. We used 
government bills, official reports, research reports, articles, and books to describe 
and analyse the different policy contexts and EQA systems. In the analysis, we 
observe shifts and continuities as institutional reproduction and change. In this 
chapter, we also discuss governing in terms of the different expectations raised by 
the national EQA systems and liken the historical expansion of the systems to an 
evaluation machinery. 


Introduction 


This chapter, in addition to forming the background for the coming chapters in this 
book, contextualises the Swedish national evaluation and quality assurance (EQA) 
systems and explicates their national reform contexts.! Higher education is an area 
that European and global policy efforts have greatly influenced in recent years. The 
Bologna declaration and the joint work on developing common indicators for 
assessing quality in higher education are far-reaching examples of such policy work. 
In parallel with other policy areas, the contemporary education policy strongly pro- 
motes the idea of systematic EQA and is part of what Power (1997) has discussed 
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as the “audit society”, Dahler-Larsen (2012) as the “evaluation society”, and Neave 
(1998) as the “evaluative state”. These European policy efforts have also influenced 
the Swedish national context. Thus, we recognise the importance of global and 
European influences (see, e.g. Ozga et al. 2011; Grek and Lindgren 2015), but in 
this chapter, we concentrate on exploring the relationship between national EQA 
systems and governing in the Swedish context. This account, we believe, can pro- 
vide insight also into how changes in other countries’ EQA systems are part of the 
contemporary governing of higher education. 

Over the last few decades in Swedish higher education, various national EQA 
systems have been decided, developed, and put into effect. Over time, these systems 
have displayed diverse political purposes and directions and exhibited different 
designs. The ramifications for the EQA systems that emerged from the policy con- 
text and the design of these systems form parts of the complex and comprehensive 
work of governing. Here, our objective is to provide a historical account and describe 
and analyse the national EQA systems for higher education, their designs from 1995 
to the 2011-2014 system, and their relation to governing. We explore the political 
process leading up to the 2016 system in the chapter “Hayek and the Red Tape: The 
Politics of Evaluation and Quality Assurance Reform — From Shortcut Governing to 
Policy Rerouting” and scrutinise this system in detail in the chapter “Re-launching 
National Evaluation and Quality Assurance: Expectations and Preparations”. 

In this chapter, the following questions directed our analysis and also organised 
our presentation of the national EQA systems: 


e What is being evaluated? Why? By whom? How? What are the consequences in 
terms of expectations? 
e What are the implications for higher education governing? 


In the following, we present a theoretical frame for our understanding of the dif- 
ferent national EQA systems that have evolved over the past 25 years. Thereafter, 
we describe the major reforms in Swedish higher education as a context for the 
design and use of the EQA systems. We divide this description into the periods the 
respective designs were in operation. Finally, we discuss the EQA systems in rela- 
tion to governing by expectations and inductively based on Dahler-Larsen’s (2012) 
idea of evaluation machines (see next section). 


A Theoretical Approach to Evolving National Evaluation 
Systems 


As stated in the first chapter, we understand governing as a verb and emphasise the 
work carried out in different ways by different actors in diverse places and spaces 
and by various means to reach certain aims (Clarke 2015). Policy intentions and 
aims are expressed in several policy documents and reforms, and for EQA systems, 
such intentions are also embedded in their designs or “infrastructures of rules” 
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(Fourcade 2010, pp. 571-572). The designs prescribe what is to be done (what is to 
be evaluated, how, by whom, and for what reasons [what is hoped to be achieved]), 
orient the attention of those taking part in these processes, and influence their behav- 
iours, activities, and perceptions of their enterprises (Dahler-Larsen 2014; Segerholm 
2001). 

For the purpose of this chapter, we recognise the dynamic relationship underly- 
ing both institutional reproduction and change (Mahoney and Thelen 2010). Change 
does not necessarily need to encompass the whole institution but can target and 
influence parts of it. EQA systems may change gradually or more dramatically, and 
these dynamics hold implications not only for governing but also for our under- 
standing of expectations of what should count as, for instance, “good quality” 
(Hopmann et al. 2007). Thelen (2003, p. 213) argues that models of path depen- 
dency, accompanied by change as rapid and drastic punctuations, need to be com- 
plemented by other tools that enable us to account for a more gradual and dynamic 
relationship, as these processes may be more incremental than usually proposed. 
Departing from such an understanding of stability versus change, we use a set of 
concepts that capture this dynamic in different ways. 

First, conversion, marking a change originating from within the institution itself 
when existing frameworks come to be enacted in various ways, resulting in institu- 
tions’ reorientation towards new goals or missions. If new rules and procedures that 
originate from outside the institution are put into place, the concept of displacement 
refers to both more radical exogenous alterations, as well as slow, incremental pro- 
cesses of change. Institutional layering marks a gradual process of change, involv- 
ing revisions, additions, and modifications when new elements are added to and put 
alongside old ones. Each revision may be small, but when placed alongside one 
another, they accumulate and result in a fundamental change. Institutions also face 
the risk of drift, erosion stemming from an incapacity to respond to the external 
context. Another such risk is exhaustion by slow-moving breakdown, resulting in 
self-destruction from within (Mahoney and Thelen 2010; Thelen 2000, 2003). We 
use these concepts as analytical descriptors of the evolving national EQA systems. 

We relate to the idea of a governing — evaluation — knowledge nexus in that the 
notion of “evaluation machines” (Dahler-Larsen 2012, pp. 176-182) is used as a 
basis to discuss the expansion of the EQA systems over time. Evaluation systems 
may be conceived as evaluation machines, since they are being based on “distinctive 
epistemological perspectives”, “organisational responsibility”, “permanence”, and 
“focus on the intended use of evaluations” (Dahler-Larsen 2012; Leeuw and Furubo 
2008, pp. 159-160). We will elaborate on the evaluation machine metaphor further 
in the discussion. 

Our account is based on documentary materials, such as government bills, offi- 
cial reports, and descriptions of the EQA systems, by the two national agencies 
responsible for higher education during the periods. We also use secondary sources 
such as research reports, articles, and books that describe and analyse the different 
policy contexts, and EQA systems in higher education in Sweden. 
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Shifting Policy Contexts: Continuities and Shifts in Evaluation 
Designs 


In this section, we present continuities and shifts in Sweden’s higher education poli- 
cies, along with the variety of national EQA systems designs that emerges from 
1995 to the system decided in 2016. We find that descriptions of the major reforms 
are contextual information that is necessary for understanding the EQA systems’ 
designs. The periods slightly extend into each other since the termination of one 
system overlaps with the introduction of a new one. Before we go into this presenta- 
tion, we offer a short background on the history of EQA in Sweden. 


-1994 


External evaluation is a rather new phenomenon within the landscape of Swedish 
higher education institutions (HEI). From 1477, the formation of the first university 
in Uppsala, up until the 1960s, institutionalised forms of external evaluation did not 
exist. According to Gréjer (2004; see also Neave 1998), external evaluation was a 
response to problems related to the expansion of higher education, and to the trans- 
formation from an elite institution to a mass university. From 1950 to 1960, the 
higher education system expanded from 16,400 students to 37,000, and by 1970, 
the number had increased to 130,000 (Gröjer 2004, p. 50). This expansion involved 
not only new groups of students but also new HEIs, staff, programmes, etc., and 
external evaluation was perceived as a state instrument to retrieve knowledge that 
could be used to plan, steer, and improve the sector according to contemporary 
utopian and = rationalistic ideas. Notably, the first national agency 
(Universitetskanslerstimbetet, UKA), which was installed in 1964 and was respon- 
sible for planning and sizing, focused its first evaluations on issues on pedagogical 
aspects such as improvement in teaching and examination. The 1970s saw the emer- 
gence of the idea that evaluation could be used at a national system level in order to 
control whether national goals were attained. Not only examination frequencies, i.e. 
output, were used, but also data on the inner workings of HEIs in terms of teaching, 
examination, students’ previous knowledge, study habits, teachers’ working condi- 
tions, etc. were acknowledged. As Gröjer (2004) noted, “effective development 
could only be achieved if the entire education system was scanned and evaluated” 
(p. 64). Thus, evaluation originally served the purpose of making the inner world of 
higher education “visible” to the state. External experts and agency staff who used 
implicit professional standards and indicators conducted the evaluations, and meth- 
ods were adapted contextually without any overall agency policy governing the 
assessments. Information from the evaluations did not follow any explicit official 
plan but was still distributed to those who were affected (Gröjer 2004). In 1977, a 
higher education reform informed by ideas on decentralisation and management by 
objectives called for further national evaluations. The new national agency 
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(Universitets- och hégskolecimbetet, UHA) engaged independent researchers and 
started building networks to facilitate exchanges of experiences and ideas. 
Conferences, seminars, and information were also used to develop evaluation 
practices. 

According to Gréjer (2004), notions of inefficiency within the higher education 
sector in the 1980s made evaluations increasingly important. Components like site 
visits, peer reviews, and criteria for international comparisons were implemented 
and methods developed. However, HEIs still held responsibility for teaching quality 
and education results, whereas EQA served to control quality at the national level. 
At the end of the 1980s, the concept of “quality” began to manifest itself within the 
language of EQA, and the perspectives of new groups of actors including students, 
student unions, and representatives from the working life (like potential employers) 
were introduced in the evaluations. The purpose of the evaluations was both to 
improve and control; however EQA was still based on implicit criteria and a group 
of directly involved experts who, based on their professional discretion, designed 
and implemented each evaluation (Gréjer 2004). Gröjer (2004) describes a continu- 
ous process of professionalization for the evaluative activities during this period. In 
this process, specific knowledge was developed, for example, during site visits 
where actors had the opportunity to learn from each other. Increasingly also, the 
national agency (UHA) argued that the HEIs must use the evaluation results for the 
purpose of improvement. 


1995-2001 


The overall aims of the higher education reform of 1993 were to increase the free- 
dom of HEIs, to establish incentives for quality development, and to improve effi- 
ciency in the use of resources in HEI activities. This reform also dramatically 
changed the preconditions for HEIs since the entire state allocation system was 
altered to a governing system based on economic incentives and productivity (Bauer 
et al. 1999; Government Bill 1992/1993:1). A performance-based funding system 
(Herbst 2009) based on a per-student state grant system was introduced, with one 
sum for each registered student and a larger sum for each student passing the course 
requirements. This meant that the previous system of applying for state grants in 
relation to the number of students was abandoned, as well as the applications for 
funds to install professorships and senior lecturers, that when approved, were 
granted by royal letters. Another novelty compared to the central planning in the 
1960s was local freedom for the HEIs to decide on educational content in their 
courses and programmes. Specific national/state requirements were however still in 
operation for professional programmes (e.g. teacher education, physicians). Internal 
quality assurance (IQA) systems at the HEIs were also introduced as a mandatory 
requirement along with a demand for obligatory course evaluations (Bauer et al. 
1999; Government Bill 1992/1993:1). 
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The design of the national EQA system during this period, it was argued, was to 
stimulate internal quality work in order to uphold and enhance quality. A new 
national agency, the Swedish National Agency for Higher Education (SNAHE, 
Hogskoleverket), was established with the commission to push and control the 
HEIs’ work with quality issues (Government Bill 1992/1993:1). In line with these 
motives, the design of the EQA system was directed at the HEIs’ internal quality 
assurance work, leaving it to the HEIs to decide how this was to be carried out. The 
SNAHE carried out two cycles of these types of evaluations (SNAHE 1998). 
Another part of the EQA system was directed towards accreditation for awarding 
magister degrees. Both these evaluation types were performed in a similar way, by 
a so-called self-evaluation based on a particular national template, peer review with 
a site visit carried out by external colleagues, plus a public report. The process as a 
whole was administered by the SNAHE. All HEIs were evaluated in 3-year cycles. 
During this period, with the 1993 years reform as a starting point, recurring external 
control of higher education through a national EQA system was introduced for the 
first time in Sweden. 


2001-2007 


No major national quality assurance reform was decided during this period, but 
some substantial changes were nevertheless made regarding the design of the 
national EQA system. The system was now said to be a means to guarantee a mini- 
mum standard in the education provided, to enhance trust in HEIs, to increase stu- 
dent influence, and to serve students with information so they can make informed 
choices (Government Bill 1999/2000:28). 

The design of the EQA system shifted its focus to be directed to quality in educa- 
tion, that is, evaluation of the quality in academic subject courses and programmes 
(Government Bill 1999/2000:28; Franke and Nitzler 2008). Another part of the 
design was to include thematic evaluations, which were directed at, for example, 
student influence and diversity. Accreditation for rewarding degrees and certificates 
and of so-called scientific areas (e.g. the right for a university college to establish 
PhD programmes and award doctoral degrees) was another part. All types of evalu- 
ations were carried out in line with a local evaluation model developed in the 1980s 
by Sigbrit Franke-Wikberg (1990), who at this time was the director general of the 
SNAHE. The model was adapted to serve a national perspective and consisted of a 
self-evaluation based on a national template. The template for the subject and pro- 
gramme evaluations asked the responsible department for information about three 
aspects: preconditions for courses and programmes, the education processes, and 
the outcomes of the education processes (SNAHE 2001, 2003). The template 
emphasised preconditions such as number of faculty with PhD, their positions, and 
number of enrolled students. The three aspects were then to be related to one another 
in an analysis and an assessment of the education the departments provided. A self- 
evaluation report was sent to the SNAHE, and the work with it was supposed to 
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engage the whole faculty in an internal discussion of their work. A group of subject/ 
programme experts carried out a peer review, and for the first time, students were 
part of this external evaluation group. The peer-review group made site visits and 
conducted interviews with department heads and managers, teachers/researchers, 
and students. The group produced a written public report in which the SNAHE 
included their decision on whether the education was assessed to be of sufficient 
quality. Hence, the reviewers had to provide a cut score that encapsulated their 
judgement and formed the basis for the SNAHE’s decision. Sanctions were also 
introduced with this EQA system, meaning that if the department/HEI did not 
improve, the right to award degrees or certificates could be revoked. A follow-up 
was therefore performed a year after in cases of a decision of inadequate quality. 
This happened very rarely (Wahlén 2012), but the entire base for HEIs to provide a 
certain level of education (course or programme) could be in jeopardy because the 
state grants regulated in the 1993 reform were (and still are in 2019) coupled to the 
right to award degrees. This EQA system was run in a 6-year cycle, and the inten- 
tion was to include all academic subjects and programmes. 

During this period, the HEIs had to evaluate their educational quality in line with 
the national template and had to yield to assessments conducted by an external 
group of “colleagues” including students. To adapt to these new circumstances and 
support departments in their work with self-evaluations, many HEIs expanded their 
administrations with new functions such as “quality officers” at the faculty level and 
deputy vice-chancellors with education quality as a particular responsibility at the 
central level (Segerholm and Åström 2007). The number of evaluations that the 
universities and departments had to engage in increased substantially with this 
rather extensive system, and there were signs that the previously existing internal 
evaluation models gave way to the national model and its templates (Segerholm and 
Åström 2007). There were also signs of evaluation influence, before an actual evalu- 
ation process had even started; for example, several signs showed that the attention 
of the HEIs was directed at what was asked for in the national template, and not 
what had locally been prioritised previously (Segerholm and Åström 2007). 


2007-2011 


The major change in preconditions for higher education during this period was the 
so-called Bologna reform in 2007. The entire structure for Swedish higher educa- 
tion was altered to include three levels (undergraduate, advanced, and graduate) 
rather than two (undergraduate and graduate). This system introduced a new order 
for degrees and certificates that required students to achieve learning outcomes to 
get a degree/certificate. Subsequently, specified learning objectives for all individ- 
ual courses were now also required. Another novelty was the establishment of the 
term “subject areas” (i.e. either an academic subject or composite of related aca- 
demic subjects) compared to the previous, more strict division in academic subjects 
(e.g. political science, sociology, and psychology). At the end of this period, the 
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government decided on a new national agency that should exclusively supervise and 
evaluate higher education, the Swedish Higher Education Authority (SHEA, 
Universitetskanslersdmbetet). 

The design of the national EQA system stayed rather much the same but with 
some stress on the relationship between the learning objectives (i.e. the require- 
ments for passing an individual course) and the learning outcomes (i.e. the require- 
ments for a degree/certificate) (SNAHE 2007). Evaluations of the IQA systems at 
the HEIs were reintroduced and followed the European Association for Quality 
Assurance in Higher Education’s (ENQA) recommendations (Ministry of Education 
2009; SNAHE 2007). The subject and programme evaluations were to be propor- 
tionate based on a simplified national self-evaluation template. Those who seemed 
to live up to the quality requirements were to be evaluated less extensively (i.e. no 
site visits). The introduction of rewards (a distinction for an eminent educational 
environment) to departments that delivered “good quality” education was a new 
feature in this period (ibid.). 

The design of the EQA system more or less emphasised the constant presence of 
external control but also directed some attention to the general idea of the relation- 
ship between (learning) objectives and outcomes. This system also represents a mix 
of sticks and carrots (i.e. threats to withdraw degrees and quality rewards) as a way 
to stimulate, or force, the HEIs to adapt to the new conditions. 


2011-2014 


Swedish higher education in the last period covered in this chapter was influenced 
by all previous reforms, which were layered on top of each other since none of them 
had been dramatically challenged or altered (Thelen 2003). There was a kind of 
incremental process towards a higher education system that became more and more 
characterised by New Public Management (see Pollitt 1995, Pollitt and Bouckeart 
2017). As an additional step in this direction, the government decided on what has 
come to be called the “autonomy reform” in 2010 (Government Bill 2009/2010:149). 
This reform concerned local freedom for the HEIs to organise internally, make deci- 
sions on types of positions and requirements for employment, and allocate resources 
internally at their own discretion. Just before this autonomy reform, the govern- 
ment, after a tense conflict with the SNAHE, decided to reform the design of the 
national EQA system (Government Bill 2009/2010:139). Its results-oriented design 
was thereafter severely critiqued (see, e.g. Kettis and Lindberg-Sand 2013); we will 
return to some of the reasons for this in the coming chapters. The SHEA’s member- 
ship in the ENQA was revoked because this system did not fully adhere to the 
ENQA statutes, and the system was finally terminated. The last evaluations in the 
system were carried out in spring 2014, and the final public reports were published 
in 2016. 

When the design of this EQA system was decided upon, it was justified in the 
policy texts by the need to increase quality in higher education (Government Bill 
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2009/2010:139). Sweden, it was argued, also needed to strengthen its international 
position in the global economy and education market. A third motive was the need 
to clarify education quality in relation to the students and to the society at large. As 
in the 2007—2011 system, it was quality in education, that is, in subject areas (the 
new term introduced earlier) that should be evaluated. Also, as in previous systems, 
accreditation for the right to award degrees and certificates was part of the design. 
The dramatic change concerned how quality in education should be evaluated: from 
a model where the relations between preconditions for education, the process, and 
the outcomes/results were the basis for assessing quality, to a new evaluation model 
decided by the government. (This decision was in itself quite unique, because the 
responsible national authority normally makes such detailed decisions.) This design 
was product oriented (Franke-Wikberg and Lundgren 1980, 1981; House 1978) and 
mainly directed to assess student outcomes as measured through the indicator of 
students’ independent projects (in the social sciences often limited empirically 
based studies presented as a small thesis/report). As before, a mandatory self- 
evaluation, in line with a national template asked for student grades, share that 
passed, etc. Quality assessment was delegated to an expert panel of peers, students, 
and representatives from areas external to the HEIs such as private companies or 
from the public sector. A sample of students’ independent projects for the bachelor, 
magister, and master degrees, the self-evaluation, and video interviews with depart- 
ment representatives, teachers, and students (instead of site visits) formed the basis 
for the assessments (SNAHE 2010, 2012, 2013). The external panel produced a 
public report including the SHEA’s decision. If the SHEA decided that the quality 
was insufficient, a plan for improvement was requested and a follow-up conducted. 
Sanctions were the same as before, but the carrots were resource allocation by state 
grants partly related to the assessment of quality (Government Bill 2009/2010:139). 
The political focus on “quality” during this period can be observed in the govern- 
ment bill (Government Bill 2009/2010:139) in which the design was proposed: The 
term “quality” alone or in connection with other terms appears eight times per page 
as a mean (Segerholm 2010), without being given any substantial meaning apart 
from student outcomes as measured by assessing students’ independent projects for 
the bachelor, magister, and master degrees. 

As we will see in the chapter “Hayek and the Red Tape: The Politics of Evaluation 
and Quality Assurance Reform — From Shortcut Governing to Policy Rerouting”, 
the design of the national EQA system in this period marks a turn in evaluation 
ideology in two important ways. Firstly, the system is deliberately based on ideas 
regarding autonomy in the sense that it steers evaluation away from preconditions 
and education processes in an attempt to not interfere with the internal work of 
HEIs. Secondly, it involved a mode of governing by evaluation in which the govern- 
ment, by rather harsh means, forced a totally different model on the HEIs — a design 
based basically on the relationship between expected learning outcomes for a par- 
ticular degree and student outcomes as measured by assessing students’ indepen- 
dent projects. The stress was clearly on what has been labelled as evaluation of 
effects (results), which were quite independent of education and/or learning process 
and preconditions. There are examples, however, where the self-evaluation reports 
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were given more weight in the evaluations. This slightly increased stress on 
self-evaluations was something that occurred during the period, much as a response 
to the criticism from the HEIs, as we understand it. The emphasis on evaluation of 
effects is also visible in the composition of the external panels, where HEI external 
representatives (or potential employers) were now for the first time acting as 
evaluators. 


Discussion 


In this chapter, we have described the major reforms in Swedish higher education 
and how the national EQA systems have been designed, used, and influenced HEIs. 
We have noted reproduction as well as change in the relatively short history of EQA 
in Sweden. Many components of systems in terms of design (e.g. peer reviews, self- 
evaluations, site visits, public reports) are introduced in the 1980s and 1990s as 
EQA was amplified on a broad national scale, and these were later complemented 
by additional components (e.g. thematic evaluations, accreditation) that were com- 
bined in different ways over time producing new aggregate system designs. We thus 
identify an overall process of expansion and change in the comprehensiveness of the 
various EQA system designs over time. The designs developed from a rather limited 
scope in direction as late as the 1990s, including evaluation of the HEIs’ IQA sys- 
tems and accreditation, over a very comprehensive design including at a minimum 
quality evaluations, thematic evaluations, and accreditation, to some intermediate 
state in the 2011-2014 period. The result of these developments in which the designs 
of the different EQA systems are layered over one another, mixing and blending 
new parts with the old and, when combined, results in a reorientation (Mahoney and 
Thelen 2010). 

Moreover, ways of organising designs by way of technologies like visibility, 
comparability, economic rewards, and sanctions that foster and trigger certain 
modes of behaviour have been added over time. Such technologies are productive 
from an organisational perspective in the sense that they make things happen, and 
they also have a radical impact on the ways individual components are employed. 
For example, if the quality of HEIs is to be comparable on the basis of public reports, 
these must be reliable and comparable to all others of a similar kind. A public report 
that is part of an EQA system and explicitly serves the purpose of comparability 
thus puts increasing demands on methods and ways of writing in terms of validity 
and reliability. Such issues related to changes in EQA designs and the implications 
of such changes on practices and behaviours will be explored more in the upcoming 
chapters of the book. 
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The issue of change is interesting in different ways. First of all, changes in national 
EQA systems do not come about by themselves; each reform involves a political 
process and agency work, to plan, design, and implement new system parts and 
ways of organising them. Change thus produces fundamental shifts in the work of 
actors directly involved in evaluations. Our results therefore raise questions about 
whether the governing potential in the EQA systems in the Swedish case partly 
relies on the shifts themselves. By constantly changing the systems, expectations 
are also changed and form one important part of the work of governing. While 
change in EQA systems serves to produce change (i.e. improvement) within HEIs 
as new aspects of practices are evaluated over time, this change arguably also pro- 
duce social acceleration (Rosa 2013) since each new national system brings about 
time-consuming efforts within the HEIs in terms of interpretation and translation 
(Ball et al. 2012) — efforts that are not only related to concrete practices of dealing 
with evaluations but also related to core activities such as organisation, planning, 
teaching, and examination. 

Overall, our account confirms Hopmann’s (2008) thesis that higher education 
over the last two decades has moved from being an internally managed “ill-defined 
problem” (evaluated by professionals themselves who needed leeway to define their 
own practice) to a “well-defined problem” that is managed and controlled by exter- 
nal (and internal) “expertise” by way of using indicators and standards. According 
to Hopmann (2008): “Expectation management changes [higher education] dramat- 
ically. The core focus shifts to more or less well-defined expectations of what has to 
be achieved by whom” (p. 424). 

Although we recognise that European (and global) EQA policy interacts with the 
Sweden’s national policy, the results also show that references regarding EQA 
design are formed rather endogenously through the social and institutional contexts 
in which the interactions are established. One observation is that the overall picture 
of the changes in the national EQA systems in terms of Mahoney and Thelen (2010; 
Thelen 2000, 2003) can be identified as institutional layering. However, over time 
there is also an ongoing process of displacement that has changed the entire direc- 
tion of Swedish higher education through the different designs of EQA systems and 
particularly the various kinds of evaluation models. Displacement here involves 
fundamental change through more active interventions in prior arrangements in 
terms of democratic ideals and the creation of new market-oriented alternatives in 
their place. Overall, this development implied a relative state suspension of ambi- 
tions to guarantee equivalence within the higher education sector. Instead, students 
were increasingly seen as consumers, and diversification in terms of education qual- 
ity was seen as a problem that could be targeted through competition. This displace- 
ment is particularly visible in the 2011-2014 system, in which increased stress was 
put on providing information to students to facilitate informed choices, informing 
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the society at large of the accomplishments of higher education in general (account- 
ability), and on including representatives of potential employers. This system was 
imposed on the HEIs, and traditional ideas on evaluation were displaced in favour 
of new components and technologies associated with new behavioural logics — 
overall, an alien approach to the higher education sector at that time. 

A second observation is that the change in evaluation models displays a succes- 
sive advancement of comparability among HEIs. Comparability as a technology 
hence serves the purpose of establishing common standards and agreements and 
organise the HEI sector within an international and national market. Taken together, 
external demands have increasingly, and bit-by-bit, resulted in a displacement, 
where HEIs become more consumer oriented, a movement provoked by the evalua- 
tion exercises. In these processes, HEIs need to precisely spell out what it is the 
students need to learn and also the HEIs’ degree of success in that respect (a decla- 
ration in advance of the quality of the service the student is “buying”, similar to 
custom charters.) Rider et al. (2013) describe the profound impacts of universities’ 
transformation from public and democratic institutions into marketised networks. 
These changes in the higher education system are similar to some of the changes 
observed in education more generally in Sweden and elsewhere (see, e.g. Ozga et al. 
2011). 


Governing by Expectations 


When considering the expectations these different EQA systems may give rise to 
and the governing role they fill, we would like to emphasise the following: 

After the 1993 reform, particularly from 1995 and onwards, there is a constant 
presence of some kind of national evaluation of higher education. This constant 
presence of external control also leads to the expectation for it to be there and to 
continue to be present. In turn, this is part of making higher education “auditable” 
according to Power (1996) or evaluable (see also Sahlin-Andersson 1995). 

In the later periods, when the designs of the EQA systems first included sanc- 
tions, and then rewards, the HEIs also developed expectations of such sticks and 
carrots. The possibility of having the right to award degrees and certificates revoked 
makes the HEIs also expect such sanctions to be used (which they also are, albeit 
rarely). The consequences of expectations of such high-stakes evaluations are well 
known in educational research, particularly concerning widespread testing, where 
phenomena such as teach-to-the-test and window-dressing are developed (see, e.g. 
Linn 2000). Compliance is another consequence that easily makes educational con- 
siderations give way to juridical or managerial ones in order to avoid criticism 
(Lindgren et al. 2012; Solbrekke and Englund 2011). 

From the implementation of the Bologna system, with Sweden’s stress on a 
rationale based on objectives and results, particularly manifested in the design of the 
2011-2014 EQA system, and its emphasis on student outcomes and attainment, the 
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state expects the HEIs to deliver students who produce independent projects that are 
assessed as good enough. Consequences of this that we know include, for example, 
changes in resource allocation so that supervisors in courses for independent proj- 
ects get more time for supervision and teachers in other courses get less time teach- 
ing (cf. Sørenssen and Mejlgaard 2014, pp. 26-27). Overall, such strategic responses 
to EQA raise critical questions. Do the designs of national EQA systems provoke 
the desire to improve/comply but take away/distort the performance? Hence, as 
pointed out by Hopmann (2008): “only those results which can be ‘verified’ accord- 
ing to the stakes given and do not meet expectations become problematic, and only 
those outcomes which meet the pre-defined criteria are considered a success” 
(p. 424). 

As noted above, accreditation has been part of all EQA systems and has stayed 
much the same over the different periods — an important continuity to acknowledge 
despite certain and other simultaneous and ongoing processes of change. The differ- 
ent national agencies (the SNAHE and the SHEA) more or less had the same expec- 
tations for what the HEIs have to show in order to get permission to offer PhD 
programmes or to get the right to award degrees and certificates. This leads to stabil- 
ity in what the HEIs expect these accreditation processes to direct attention to, and 
that is of foremost attention to certain preconditions, such as share of teachers with 
a doctorate. One known consequence of this, however, is that HEIs try more 
intensely to increase their shares of faculty with a doctoral degree when they apply 
for the right to start a PhD programme. In general terms, the consequences of these 
reciprocal expectations are that the HEIs direct resources to live up to the different 
requirements in the evaluations. 

The successive additions in the external panels of students, and of future employ- 
ers in the 2011-2014 system, teach the HEIs, and develop expectations that parts of 
society outside the higher education sector, have legitimate interests in the scrutiny 
of higher education. Expectations are also raised about them having knowledge and 
competence enough to evaluate higher education. Extended influence by external 
stakeholders is by no means a Swedish higher education phenomenon, as Deem 
et al. (2007) and Magalhães et al. (2018) show in their studies of managerialism and 
higher education governing boards. 

The state, on the other hand, expects the HEIs to accept all sorts of evaluators and 
also expects the HEIs to acknowledge that the expertise of the external panel is suf- 
ficient when it comes to higher education. A plausible consequence of these last two 
sets of expectations is a shift in the mindset of HEI managers and teachers/research- 
ers to be more receptive to external demands on the direction of their work, that is, 
to make higher education and research better adapted to market needs. For example, 
this may mean increased efforts to produce more “useful” (applied) education and 
research. This is an example of what Dahler-Larsen (2012) labels constitutive 
effects, pointing to the potential of evaluations to influence not only behaviours but 
also our perceptions of the phenomenon/activity/programme that is being 
evaluated. 
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The final kind of expectation we bring forward is based on the descriptions of the 
different designs of the national EQA systems. This expectation suggests that the 
shifts in designs themselves make the HEIs to expect changes. A consequence of 
that is that it has become necessary for the HEIs to always keep an eye on national 
policy developments and on what is required of them. They thus accept constant 
change, constant pressure, and constant control and must be on alert, thereby pos- 
sibly avoiding the risk of drift (Mahoney and Thelen 2010). Depending on the more 
detailed shifting but also stable requirements in the different designs of the EQA 
systems, governing by expectations is both about what the HEIs expect the state (the 
national agencies) to do and what the state and national agencies (through decisions 
and policy) expect to happen at the HEIs. 


Building an Evaluation Machinery? 


Overall, we contend that the historical process of establishing national EQA sys- 
tems in Sweden shows resemblance to what Dahler-Larsen (2012) describes in 
terms of evaluation machines. In this book, we use the evaluation machine analogy 
in our explorations of evaluation as a practice in governing the Swedish higher edu- 
cation case. As shown in this chapter, the EQA systems change constantly, leading 
us to use the notion of an “evaluation machinery” to denote the assemblage of ele- 
ments that we have identified during the covered period. We equal an evaluation 
machinery with Dahler-Larsen’s characterisation of evaluation machines as an ideal 
typical concept that draws attention to development within the audit society, where 
evaluation has become institutionalised and professionalised so that “arbitrariness 
and subjectivity” are eliminated (Dahler-Larsen 2012, p. 176). They are: 


[mJandatory procedures for automated and detailed surveillance that give an overview of 
organizational activities by means of documentation and intense data concentration. 
(Dahler-Larsen 2012, p. 176) 


Similar to evaluation machines, the Swedish national EQA machinery has become 
permanent and repetitive over time and functions as a producer of “streams of infor- 
mation” rather than occasional reports (Dahler-Larsen 2012, p. 177). It has become 
increasingly embedded in HEIs organisational procedures of verification and 
resource allocation; EQA is thus framed by ideas of “organizational responsibility” 
(Dahler-Larsen 2012, p. 177). As such EQA has also become a prospective rather 
than just a summative form of evaluation. Broad scales of activities related to EQA 
are “planned in advance so they can be intentionally linked to decision and imple- 
mentation process” (Dahler-Larsen 2012, p. 177). EQA is hence increasingly recip- 
rocal and has become a natural condition for HEIs. Over time, EQA has become 
based on “distinctive epistemological perspectives” and increasingly “relies on a 
number of tools or scripts such as definitions, indicators, handbooks, procedures, 
guidelines, etc., to support fairly standardized operationalisations” (Dahler-Larsen 
2012, pp. 177-178). Finally, as an evaluation machinery, EQA cover “phenomena 
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that have broad scope in time and space” (Dahler-Larsen 2012, p. 178). Higher 
education involves extensive and complex activities that are detailed “in a system- 
atic and integrated way that permits comparison among areas of activities” (Dahler- 
Larsen 2012, p. 178). 


Finally 


The notion of an evaluation machinery harbours a range of aspects that will be 
explored further in the upcoming chapters of the book. This notion draws attention 
to the role of documentation and specific forms of documentation in terms of self- 
evaluations that have become institutionalised over time. An evaluation machinery 
also require distinctive roles and knowledge for their functioning. They must be 
designed, engineered, and operated (Dahler-Larsen 2012). For example, what are 
the implications of increasing demands on external assessment panels and site visits 
in terms of forms of knowledge, expertise, experience, and social competence? 

The issue of constitutive effects (Dahler-Larsen 2012) will also be pursued as an 
important theme in the book. In some of the following chapters, we will look more 
closely into this and explore some of the above described national EQA systems and 
processes, their consequences, and the way they influence and govern higher 
education. 

In the next chapter, however, we turn to the wider international context and situ- 
ate the Swedish example within Europe in order to understand development in 
higher education policy and EQA systems. 
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Abstract The increased importance given to European policy concerning evalua- 
tion and quality assurance in higher education in the Swedish national policy con- 
text is explored in this chapter. The description rests on interviews with what here is 
labelled policy brokers and on the material from the European Association for 
Quality Assurance in Higher Education (ENQA), as well as government bills, par- 
liamentary minutes, national agency reports, and university teachers’ union jour- 
nals. The results show that ENQA membership requirements played a significant 
role in the Swedish policy debate on the design of the 2016 national evaluation and 
quality assurance system. Dissemination channels between Europe and Sweden are 
populated by individuals with similar functions and positions, e.g. that ministers 
often meet ministers. Within Sweden, European policy is disseminated by and 
through individuals who move between different positions within the ministry of 
education, national agencies, and higher education institutions. Different 
organisations also communicate with each other within Sweden, ensuring European 
policies reaching into higher education institutions. Such European governing 
attempts are carried out in activities like networks, conferences, papers, guidelines, 
and by using different forms of knowledge, both inscribed, embodied, and enacted. 


Introduction 


In the previous chapter, we analysed the development of national evaluation and 
quality assurance systems mainly as a national issue. However, we also noted that 
Sweden’s engagement in the Bologna Process influenced the evaluation and quality 
assurance systems, for example, by placing an increased stress on expected learning 
outcomes. The European policy agenda also gained power in the debate following 
the membership status of the Swedish National Agency for Higher Education 
(SNAHE) in the European Association for Quality Assurance in Higher Education 
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(ENQA) following the 2011-2014 national system. The amplified importance of 
European evaluation and quality assurance policy for Sweden is portrayed in this 
chapter. We will deepen the study of European influence and governing potential 
and explore the European arena and its relation to Swedish higher education policy 
concerning evaluation and quality assurance (EQA).! 

Knowledge and ideas about evaluation and quality assurance may exist in 
national/state agencies to a lesser or larger degree. As shown in the previous chapter, 
Sweden has used a handful of national evaluation and quality assurance systems for 
higher education during the last decades, and, through these processes, certain 
knowledge about EQA and governing has been acquired (e.g. Franke and Nitzler 
2008; Kettis and Lindberg-Sand 2013). Such knowledge has progressively been 
brought together and circulated in transnational spaces (e.g. the ENQA 2010). 
Through these transnational spaces, circulation or dissemination of policies and 
knowledge about EQA among national/state agencies and contexts also takes place. 

Since the signing of the Bologna agreement for European higher education, a 
number of initiatives, programmes, and organisations have been launched. These 
efforts can be summarised by the concept of “Europeanisation” (e.g. Grek and Lawn 
2012; Jacobsson 2010a). An explicit political purpose is to safeguard Europe’s 
position on the global market. Education is believed to promote this, and it is argued 
that student, teacher, and researcher mobility would facilitate this ambition 
(Standards and Guidelines for Quality Assurance in the European Higher Education 
Area, ESG, 2015, p. 6). Coordination of different national education systems is one 
way to sustain this mobility ambition. As higher education is not regulated by 
European laws and regulations, processes of coordination must rely on “soft” 
governing (Lawn 2006) through means like the open method of coordination, which 
is based on: 


-jointly identifying and defining objectives to be achieved (adopted by the Council); 
-jointly established measuring instruments (statistics, indicators, guidelines); 
-benchmarking, i.e. comparison of EU countries’ performance and the exchange of best 

practices (monitored by the Commission). (EUR-Lex n.d.) 


It is also said that to support and increase mobility, national higher education 
systems have to be trusted to deliver high-quality education, and it is claimed that 
quality assurance (QA) is the means to achieve this (ESG 2015, p. 6). In the last 
decades, a number of reforms have been implemented, and multiple QA activities 
have been performed (Maassen and Stensaker 2011). 

One of these initiatives and from 1999 specifically directed at ensuring “more 
comparable, compatible and coherent systems of higher education in Europe” is the 
European Higher Education Area, (EHEA n.d.-a). A central purpose of the Bologna 
Process, and hence the EHEA, is to “encourage European cooperation in quality 
assurance of higher education with a view to developing comparable criteria and 
methodologies” (EHEA n.d.-b). For this reason, the European ministers of education 


'The chapter is based on two conference papers that have been revised and updated: Segerholm 
and Hult (2015, 2017). 
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agreed to support the Standards and Guidelines for Quality Assurance in the Higher 
Education Area (ESG) in 2005, which was drafted by the European Association for 
Quality Assurance in Higher Education, ENQA (ENQA 2009; Thune 2010), and 
revised in 2015 (Standards and Guidelines for Quality Assurance in the European 
Higher Education Area (ESG) 2015). 

In spring 2019, 50 QA agencies in 28 countries were full members of the ENQA, 
which means that they met QA policies and practices set up by the organisation. 
Since all EQA activities are part of governing work (Ozga et al. 2010), our argument 
is that ENQA membership (or striving for ENQA membership) can be significant 
when attempting to understand how certain governing policies and practices are 
brokered and translated into the European context and beyond. 

In this chapter, we investigate the significance of membership in the ENQA and 
its relation to the governing of higher education. Here, Sweden is an interesting 
example in that its national agency for higher education was a full member of the 
ENQA, but this status was questioned in 2012 (ENQA 2012), and membership was 
revoked in February 2014 (ENQA 2014a). This state of affairs fuelled an already 
existing debate about the shortcomings of the national EQA system in existence 
from 2011 to 2014 and was linked to political discussions about the design of a new 
EQA system, which we analyse more thoroughly in the chapter “Hayek and the Red 
Tape: The Politics of Evaluation and Quality Assurance Reform — From Shortcut 
Governing to Policy Rerouting”. The specific focus of this chapter is the relation 
between the ENQA’s policy and Sweden’s policies on EQA and the channels by 
which European policy enters Sweden. We recognise that policy dissemination and 
circulation (i.e. brokering) may occur in bi- or multidirectional processes, not just 
from one part or level to another. The special situation of the Swedish agency’s non- 
ENQA membership did however produce a particular national policy context in 
which European and ENQA policy became very influential. Our aim is, therefore, 
to explore the influence of the ENQA policy on the Swedish national EQA policy in 
higher education and how the ENQA policy enters the Swedish policy context and 
forms part of the governing of Swedish higher education during a period of 
nonmembership of the Swedish agency. 

The questions we pursue are: 


e How is the ENQA’s policy visible in the Swedish national policy process? 

e Through what channels was ENQA policy disseminated, and what policy was 
perceived as important in Swedish national policymaking? 

e How does ENQA policy relate to the governing of higher education in Sweden? 


Theoretical Notes, Methods, and Material 


In this book, we have emphasised governing as a verb, i.e. to focus on activities and 
the work that is “done”, be it by talking to people, writing texts, arranging and 
participating in meetings and workshops, and/or by decision-making bodies. In this 
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context and as we outlined in the chapter “Governing by Evaluation: Setting the 
Scene”, we conceptualise “governing” as a verb so as to underline that attempts to 
develop, disseminate, and implement policy require activities and work, as do 
attempts to develop and disseminate practice. 

In an effort to promote stronger and more unified education policies throughout 
Europe, different activities are carried out to influence member countries (and 
perhaps others) to accept and also develop similar policies and practices. As we will 
show in this chapter, different methods are used to increase the governing power of 
European policies. Ways of disseminating policy and practice that can influence 
(govern) beyond the central European sphere have been described and analysed by 
Jacobsson (2010a, b) through the concepts of “regulative, inquisitive, and meditative” 
governing activities. “Regulative” refers to activities like rule-making and mandatory 
obligations where penalties can be invoked if the rules are broken. “Inquisitive” 
activities require organisations to be open to examination, inquiry, scrutiny, and/or 
different sorts of assessments. Finally, “meditative” activities, which form the 
foundation of the other two, are in themselves a separate type of activity that 
includes verbal and written interaction, sometimes in spaces particularly aimed for 
such activities (e.g. workshops, meetings, conferences) where the exchange of 
experiences and probing of common issues are central (Jacobsson 2010a, pp. 4-6.) 
This conceptual frame also points to different ways that policy may be brokered and 
disseminated. 

The importance of national and local contexts in these processes (Ozga and Jones 
2006; Steiner-Khamsi 2004; Sassen 2007) is also noted. European education policy 
has to make national sense and be translated and interpreted to fit specific national 
contexts so as to influence already existing policy and practice. In European govern- 
ing and policy work, nation-states are more or less actively involved. Involvement 
requires balancing internal national interests with national and international require- 
ments. To come to international agreements and consensus on ENQA membership 
requirements, active work is needed by nation-states, sometimes with actors holding 
oppositional views (see Sassen 2007) where struggles and conflicts occur over time 
and have to be addressed. Therefore, national and local contexts affect when, how, 
and what policies travel, how they are disseminated and translated, and how govern- 
ing influenced by international (European) policies is possible. In the Swedish case, 
several actors take part in discussing higher education policy and the EQA system(s), 
and the state (government and agencies) has interacted with these actors in different 
ways during the periods we studied (see also the chapter “Hayek and the Red Tape: 
The Politics of Evaluation and Quality Assurance Reform — From Shortcut 
Governing to Policy Rerouting”; Lindgren and Rönnberg 2015). There are also 
actors that function as links, or brokers, between Sweden and the European/global 
space. These are what Lawn and Lingard (2002) call “policy elites”, composed of 
actors like ministers, high-ranking civil servants, and experts in different policy 
fields. They meet in different organisations to discuss mutual policy issues, exchange 
experiences, and decide on common policies and actions. 

This chapter is based on interviews with ten such national policy brokers carried 
out mainly in spring 2015. We also collected and analysed a variety of documents 
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from the ENQA and other European organisations, from Swedish government bills 
and parliamentary minutes, documents, etc., and documents from the two national 
agencies responsible for supervising higher education in Sweden. In addition, we 
also analysed material from the university teacher union’s journal and daily national 
press (Svenska Dagbladet). 

Next, a description of the ENQA is presented based on official ENQA docu- 
ments. We then scrutinise the ENQA influence through its membership require- 
ments and relate them to the Swedish EQA contexts, showing that European 
governing attempts through QA policies enter the national context of Swedish 
QA. In this account, we also insert examples of the internal national debate in 
Sweden to exemplify the influence of the ENQA on the national policy space. Next, 
we investigate how these Europeanising processes are experienced and handled by 
what we here call national policy brokers and through which channels European 
policies enter Sweden. We then describe these intranational dissemination channels. 
The chapter ends with a discussion of our findings, which we discuss in relation to 
the importance of governing by comparison, in the dissemination of EQA and 
governing policy. 


The ENQA 


The history of the ENQA goes back to the beginning of the 1990s when “national 
quality assurance agencies had started to arrange unofficial meetings in the early 
and mid-1990s in order to exchange information and, quite practically, to get to 
know each other” (Ala-Vahalé and Saarinen 2009, p. 90; see also Kauko 2012). This 
was a process in which Sweden was actively involved, and the country also became 
one of the early members in 2000. The European Commission had a general interest 
in supporting QA processes and took a particular interest in such activities in higher 
education, leading to a number of pilot projects (Enders and Westerheijden 2014; 
Kristensson 2010). After the finalisation of these projects, ministers for higher edu- 
cation in EU and EFTA/EEA countries, representatives for QA agencies, and asso- 
ciations for higher education met in Brussels for a conference where the ENQA 
network was launched (Thune 2010, p. 9). In 2011, it was transformed into the 
organisation European Association for Quality Assurance in Higher Education and 
kept its acronym ENQA. The stated mission of the organisation is to contribute to 
quality enhancement of European higher education, to develop QA processes, rep- 
resent its members internationally, influence policymaking at the European level, 
promote cooperation among QA agencies, and “foster the European dimension of 
quality assurance of higher education” (ENQA statutes 2015, article 3). This is to be 
achieved by providing services and expertise, opportunities for networking, and 
coordinating external reviews of QA agencies. The ENQA also engages in the 
Bologna Process in projects, publication of reports and policy papers, and in coop- 
eration with stakeholder organisations in higher education in Europe and worldwide 
(ENQA statutes 2015, article 4). European EQA policy travel through these 
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activities, and are some of the means in operation to influence European countries. 
Activities like these are what Jacobsson (2010a, b) calls meditative processes. 
Another avenue for the ENQA to influence higher education through quality assur- 
ance policy and practice is the importance put on membership requirements. 


The ENQA’s Membership Requirements and Sweden 


To become a member and have access to the ENQA’s activities, member organisa- 
tions must meet certain conditions (ENQA statutes 2015, article 5-8). There are two 
types of involvement: membership and affiliation: 


Membership is open to European quality assurance bodies in the field of higher education 
that are conducting quality assurance activities as understood in the Standards and 
Guidelines for Quality Assurance in the European Higher Education Area (ESG) and act in 
compliance with the membership criteria. (ENQA statutes 2015, article 5) 


The ESG is divided in three parts. The first part concerns quality standards and 
criteria for higher education institutions. The second part “basically required QA 
agencies to check how higher education institutions had implemented Part I, and in 
Part III required the QA agencies to be regularly reviewed themselves” (Enders and 
Westerheijden 2014, p. 173). Affiliation concerns QA bodies that cannot or do not 
wish to become members. Nonetheless, they still must agree to abide by the statutes 
of the ENQA, which in Jacobsson’s (2010a, b) terminology would be considered 
governing by regulative activities. 

ENQA regulations require member agencies to undergo an external review at 
least once every 5 years, which is an example of an inquisitive activity (Jacobsson 
2010a, b). The SNAHE had been a member of the ENQA since 2000 and was also 
confirmed as a full member in 2005-2006 (SNAHE 2006). However, the EQA 
system established in 2011 was reviewed by an ENQA review panel in 2012 and 
assessed to be “fundamentally at odds with the ESG” (SNAHE 2012, p. 23). The 
main areas where the Swedish system deviated from ENQA principles were in not 
evaluating higher education institutions’ internal QA systems and in not giving 
recommendations for improvement. Also, according to the review panel, the 
Swedish agency responsible for QA could not be considered independent (ENQA 
2009, 3.6) “due to the extent to which their procedures and methods, as well as 
overall aims and objectives have been dictated by government” (SNAHE 2012, 
p. 23). This news was highlighted in the University World News, which described 
the distress it caused in the higher education sector (Mycklebust 2012). Later the 
same year, the ENQA board decided that the Swedish agency was a “full member 
under review”, and it was to be reviewed again in 2014 (ENQA 2012). However, in 
2014, the criticised national EQA system was about to be phased out, and a new 
system was under development, and the application for full membership was to be 
postponed until the new national EQA system was in place. Since February 2014, 
the Swedish agency is not a member of the ENQA (ENQA 2014a). 
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The national EQA system was critiqued, debated, and analysed by scholars 
across disciplines throughout the period of its existence of 2011-2014 (see, e.g. 
Adamson 2013; Ericsson 2014; Haikola 2013; Kettis and Lindberg-Sand 2013; 
Lundmark and Sjölund 2012; Sedigh 2013). It was challenged by the Association 
of Swedish Higher Education Institutions (ASHEI, SUHF Sveriges universitets- 
och hégskoleférbund) and debated within the Swedish Association of University 
Teachers and Researchers (SAUTR, Sveriges Universitetsldarares Förbund SULF) 
and by the Swedish National Union of Students (SNUS, Sveriges Förenade 
Studentkårer SFS). In these debates, the lost ENQA membership and suspension 
of the Swedish national agency (Swedish Higher Education Authority, SHEA) 
were used as arguments to reform the national EQA system. In an interview, one 
teacher union representative said the suspension was “embarrassing” (Dagens 
Arena 2014). The chairman of the national student union wrote that “Sweden can- 
not apply for new membership for another two years, which is a big blow to the 
international reputation of the Swedish education system” (Arroy 2014). The issue 
of ENQA membership was also brought up by the political opposition in parlia- 
ment, and the Minister of Education from the Liberal Party framed his response 
this way: 

Mister President! Jabar Amin [MP from the Green Party] has asked me what I intend to do 

to guarantee that Sweden will have a quality assurance system that is good enough and may 

install confidence for Sweden in this respect. Ibrahim Baylan [MP from the Social 

Democratic Party, our clarification] has asked me what I have done to avoid a termination 

of the membership [in ENQA] and what measures I plan to take in order to reinstall 

confidence for the Swedish higher education system. (...) The SHEA is responsible for a 

development work where they, along with representatives for the higher education 

institutions, the students, and labour market undertake certain adjustments of the quality 


assessment system. With these adjustments, the SHEA should be able to fulfil the ENQA’s 
requirements for full membership. (Parliament Minutes 2013/2014:81, p. 3) 


In 2013, the SHEA was commissioned to develop a revised national EQA sys- 
tem, which was planned to be presented publicly in spring 2014 but was cancelled 
just prior to launch (SHEA Central staff). The ASHEI sent a proposal in April 2014 
to the Ministry of Education in which they suggested a new EQA system for higher 
education where the responsibility for performing QA and evaluations would be 
transferred from the national agency to the higher education institutions themselves 
(ASHEI 2014). This was rapidly followed by an article in one of the national news- 
papers, signed by all vice chancellors and the chairman of the national student 
union. In the article, an overview of the proposed new system was presented 
(Svenska Dagbladet 2014). It was pointed out that the suggested system would be 
fully compatible with the ENQA membership requirements. An immediate response 
from the government was to assign a special investigator to develop and propose a 


>This is an organisation in which the vice chancellors of all higher education institutions form 
a strong interest group that increasingly put pressure on the government and try to develop 
common advice for how to act in relation to the new policy landscape that opened up by recent 
reforms (ASHEI n.d.). Even though the ASHEI represents the name of the entire higher educa- 
tion sector, the organisation does not necessary represent teacher and researcher interests and 
opinions. 
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new national EQA system (Government Office 2014). The instructions said that 
“principles for quality assurance at the European level shall be taken into account” 
(Government Office 2014, p. 1). 


European Policy Enters a Proposed National EQA System 


In April 2014, the government appointed the soon-to-be university chancellor and 
Director General of the SHEA as the special investigator. The instructions concerning 
European-level principles stated that the new system should include the higher 
education institutions’ (HEI) own work on QA and that the system should be quality 
driven and give the HEIs recommendations to improve education (Government 
Office 2014, p. 1). 

As will be shown in the chapter “Hayek and the Red Tape: The Politics of 
Evaluation and Quality Assurance Reform — From Shortcut Governing to Policy 
Rerouting”, the preparation with the special investigator’s proposal involved a large 
number of stakeholders (see also the chapter “Enacting a National Reform Interval 
in Times of Uncertainty: Evaluation Gluttony Among the Willing”). At this time, 
new ENQA standards and guidelines were proposed but not decided on, although a 
preliminary version from September 2014 had been presented (ENQA 2014b) and 
was taken into account. The special investigator presented her suggestion to the 
government in late 2014. Afterwards, in March 2015, the government crafted a 
memorandum through which their ideas concerning a new national EQA system 
was circulated to the HEIs and other stakeholder organisations for comments 
(Ministry of Education 2015). The memorandum was said to outline an overall 
framework, and the SHEA was then responsible for developing and implementing 
the details and design of this new system. In this work, the agency was to pay atten- 
tion to the government’s considerations as well as to national laws and ordinances. 
The agency was also to adhere to the international principles for QA that were devel- 
oped within the framework of the Bologna Process (Ministry of Education 2015). 

Throughout the memorandum, the importance of developing the system in accor- 
dance with European and international QA principles was put forward, and the 
Ministry clearly referred to the ESG and to the organisations involved in and 
responsible for them (Ministry of Education 2015, pp. 11-12). An overall emphasis 
was also put on an EQA system that should help improve higher education and not 
only control for results. 

In the following political process, the European policy on EQA for higher educa- 
tion continued to be prominent in the Swedish context. The government collected 
comments from the HEIs and other stakeholders during 2015 and produced a gov- 
ernment petition (2015/2016:76) to the parliament in December 2015 in which it 
once again was stressed that a new national EQA system for higher education should 
take into account “those principles that have been put forward within the frames of 
the Bologna process” (Government Petition 2015/2016:76, p. 7). This was also 
endorsed by many of the stakeholders in their comments on the government memo- 
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randum (Ibid.). To provide background for the position taken, the government peti- 
tion described the political aims with the ESG and the ENQA and what had happened 
to the Swedish agency’s membership in that organisation (Ibid.). 

The Parliament Standing Committee on Education handled the issue further, and 
in its discussions of the government petition, it also brought forward the European 
principles for QA (Parliament Standing Committee on Education 2016, pp. 17-18). 
The committee’s proposal to the parliament also accounted for the Swedish agen- 
cy’s membership status in ENQA and described the process that led to the termina- 
tion of ENQA membership (Parliament Standing Committee on Education 2016, 
p.14). In March 2016, the parliament finally decided on a new national EQA system 
in line with the framework set out by the government (Parliament Decision 
2015/2016:155). 

While they were waiting for a definitive parliamentary decision to launch a new 
national EQA system, the SHEA began developing and testing some of its planned 
components, some of which were already suggested in the 2015 Ministry 
memorandum (for instance, in the format of pilot evaluations of PhD programmes). 
This work continued in 2016 and 2017 when the SHEA performed pilots of the 
main parts of the new EQA system (SHEA 2016a, 2017, 2018). The new and 
existing system and its design and implementation process are explored in more 
detail in the chapters ““Re-launching National Evaluation and Quality Assurance: 
Expectations and Preparations” and “Re-launching National Evaluation and Quality 
Assurance: Governing by Piloting”. In the context of this chapter, however, we want 
to highlight that the SHEA repeatedly and consistently underlined the importance of 
the ESG in all its activities related to developing the design of and pilots of the new 
system. As an illustration, this was evident in the SHEA Guidelines for HEIs’ self- 
evaluations and external assessor panels in the pilot evaluation of institutional 
reviews that should assess the HEIs’ internal QA systems (SHEA 2016b). In those 
guidelines, particular standards and the ESG as a whole were referenced in detail. 
Occasionally, there were translations almost to the letter, as is the case concerning 
the ESG 1.2 and 1.3 (ESG 2015, pp. 11-12) and their equivalents, Aspects 3.1 and 
3.2 in the SHEA Guidelines (SHEA 2016b, pp. 22-25). 

As we have shown so far, the national political context matters (cf. Ozga and 
Jones 2006) when it comes to how receptive Sweden has been in relation to European 
EQA policy. When the Swedish agency’s ENQA membership status was finally 
revoked in 2014, concerns were raised from many stakeholders pointing to the lack 
of international legitimacy of the Swedish EQA system. One way of coming to 
terms with these national concerns was to develop a new national system more in 
line with the ESG. Our data recurrently display how policymakers and stakeholders 
in the higher education sector were preoccupied with regaining international 
legitimacy through the adoption of the ESG and reinstated ENQA membership. 
This is evidenced by continual references to and arguments for the importance of 
adherence to European policies in matters that were voiced in the national political 
debate. This is what Sassen (2007) refers to as the important policy work necessary 
for national agreement on issues where oppositional views exist. Such work is also 
needed to facilitate policy translation work to fit a specific national context (Ozga 
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and Jones 2006; Steiner-Khamsi 2004; Shattock 2014). We will get back to these 
issues in the chapter “Re-launching National Evaluation and Quality Assurance: 
Expectations and Preparations” where the design work of the 2016 system is further 
scrutinised. 

We have also shown that the European EQA policy influenced Swedish national 
EQA policy, later resulting in the new national EQA system of 2016. Some of the 
explanations for this state of affairs were also given by what we call the policy 
brokers or elites (c.f. Lawn and Lingard 2002). Next, we turn to these policy brokers 
and their experiences of how and what policies travel with an emphasis on the routes 
or channels policy takes in entering and circulating in Sweden, what we here call 
dissemination. We use the concept of dissemination to denote an intentional process 
of negotiating and spreading policy to different (European) countries and to different 
actors within countries. We also conceive dissemination to be akin to circulation, 
meaning that some ideas are spread widely and become authoritative by such a 
process (Sahlin and Wedlin 2008). The idea of channels developed from empirical 
data where some organisational relations turned out to be rather prominent. This 
inspired us to view dissemination of EQA policies through these channels. 


EQA Policy Dissemination 


In the following section, we first describe the European and other organisations’ 
links to Sweden before moving on to describe the internal Swedish dissemination 
channels. The types of dissemination activities and the primary content of those 
activities are also reported. 


European and Other Dissemination Channels 


Overall, the channels through which European EQA policies for higher education 
travel, are disseminated, and enter Sweden are generally bi- or multidirectional. But 
it is quite clear that the predicament with the questioned, and later revoked, ENQA 
membership made Swedish policy actors and brokers more receptive and sensitive 
to European and ENQA influences, giving the policy flow a more unidirectional 
character during this period. 

The ten policy brokers we interviewed worked in the Ministry of Education, the 
SHEA, the ASHEI, teachers and student unions, and a lobby organisation. Many of 
these persons had experiences from different parts of the higher education sector, 
including political and/or administrative offices and/or high-level leadership 
positions in HEIs. 

The policy brokers talked about a number of European, global, and Nordic 
organisations as particularly important for them and the organisations they 
represented in relation to EQA (see Fig. 1). Not surprisingly, informants at the 
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Fig. 1 From left to right: European/international to Swedish dissemination channels; inter- 
nal Swedish dissemination channels 


SHEA were involved with the highest number of international organisations, since 
the SHEA as a national agency is responsible for the supervision of higher education 
and is expected to engage in EQA policy and practice and promote such activities. 
Important organisations for these policy brokers were the ENQA (emphasising the 
ESG), the European Consortium for Accreditation in Higher Education (ECA), and 
the European Quality Assurance Forum (EQAF). The International Association of 
Universities (LIAU) and the International Network for Quality Assurance Agencies 
in Higher Education (INQAAHE) were put forward. As non-European organisations, 
the Nordic Quality Assurance Network in Higher Education (NOQA) and the 
Nordic Council of Ministers were also stressed. 

Another central policy actor when it comes to higher education policy and EQA 
is, of course, the government and its Ministry of Education. In our interviews, 
Ministry participants mentioned the European Commission and the Council of 
Ministers and the Nordic Council of Ministers as important venues for Sweden 
concerning policy information and discussions about EQA policy in higher 
education. Again, this is hardly surprising, as these channels are the formal ones one 
would expect European policies to take. 

Ever since the so-called autonomy reform in Sweden was decided upon in 2010, 
leaving space for the HEIs to decide on internal organisation and positions (see 
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the chapter “National Evaluation Systems”), the ASHEI has increasingly worked to 
establish a common policy platform for the Swedish HEIs and has strengthened its 
influence on national policy. This actor organises the vice chancellors of higher 
education and has a task force solely working with issues of EQA. The ASHEI 
participants claimed that their most important international contacts in this respect 
were with the European University Association (EUA), and, as the ASHEI in a way 
represents the HEIs, respondents indicated that the EUA is also an essential contact 
for the HEIs. 

Another policy actor is the Centre for Business and Policy Studies (CBPS, 
Studieförbundet Näringsliv och Samhälle SNS), which organises private companies, 
research organisations, public enterprises, and national agencies. It engages in 
policy activities, research, lobbying, and policy advocacy concerning all areas of 
societal welfare. Our informant at this organisation said that the main policy contacts 
for them, in relation to EQA and higher education policy outside Sweden, are the 
European Parliament, BusinessEurope,* and the Organisation for Economic 
Co-operation and Development (OECD). 

Two policy brokers in our sample representing teachers, researchers, and stu- 
dents in higher education were from the SAUTR and the SNUS. The informant at 
the first organisation brought forward the Education International’ (EI) as an impor- 
tant link for them in relation to EQA. The second recognised the European Students’ 
Union (ESU) as an organisation that they interact a lot with. The EHEA was also 
mentioned as a more general resource for their policy work. 


What Policy Is Important for Sweden? 


We also found it interesting to take a closer look at which issues, according to the 
policy brokers, some of the European actors disseminate in Sweden. Not surprisingly, 
the ENQA and its interest in QA models and systems were most often mentioned by 
the policy brokers. They indicated that the ENQA offers a possibility to review QA 
models in other countries and compare them to the Swedish model, thereby making 
it possible to assess the Swedish model and to monitor the changes in EQA policy. 
The policy brokers said that the ENQA paves the way for international agreements 
like the ESG and is important for mutual recognition of different kinds of higher 
education systems in European countries. This is significant for future student 
exchanges among European countries, they explained. Additionally, the ENQA 
provides opportunities to reconsider boundaries between politics and academics 


3“BusinessEurope is the leading advocate for growth and competitiveness at European level, 
standing up for companies across the continent and campaigning on the issues that most influence 
their performance” (BusinessEurope n.d.). 

‘ET is a global umbrella organisation for labour union organisations for teachers and researchers in 
preschool up to higher education. 
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and can help politicians understand that they have to work with the academic com- 
munity and not against it, as one informant put it. 

Another often mentioned association was the EUA, and the topic that was most 
interesting for Swedish policy was different aspects of autonomy for the HEIs in 
relation to central state power and authority. EUA conferences and study visits are 
argued to give insight opportunities to compare Swedish HEIs with other European 
HEIs. This comparison has, according to the policy brokers, shown that the 
autonomy for Swedish institutions is more restricted compared to other European 
countries. The HEIs in other countries are claimed to be more trusted than the 
Swedish ones, and in these countries, the HEIs also take a greater responsibility for 
their internal quality. One of the policy brokers said: 


The great boundary line, I think, is between the systems that trust the universities and the 
ones that don’t trust the universities. Sweden has had a system where they don’t trust the 
universities; we have had that for many, many years in different shapes. I hope for a change 
towards a system that trusts the universities. If you make this distinction, the result will be 
two completely different models. (PB4) 


Different European organisations also nourish conflicting interests in Sweden. 
The ESU is argued to be important in their resistance to the OECD’s instrumental 
views of higher education—that education primarily should serve the labour mar- 
ket—according to the SNUS informant. Then again, the representative for the CBPS 
emphasised BusinessEurope in their mission of emphasising the importance of edu- 
cation and training in relation to the need for the labour market and public welfare. 

Taken together, dissemination of EQA policy runs through channels with the 
help of a variety of activities that are designed to bring people and organisations 
together. In our interviews, such activities were described as conferences, work- 
shops, annual membership meetings, formal meetings, study visits to other coun- 
tries, hearings, and dialogues. Several of these are face-to-face activities and were 
talked about with enthusiasm by the informants. They resemble what Jacobsson 
(2010a, b) calls meditative activities and very much rely on social skills to be con- 
vincing enough to inspire travel and translation into different national and local 
contexts. 


Internal Swedish Dissemination Channels 


Within Sweden, the picture is more complex, indicating interesting connections 
between different organisations and actors that might aid further dissemination (see 
Fig. 1). Starting with the Ministry of Education, the informants there said that they 
“naturally” interact with the SHEA. Other connections of importance are the 
ASHEI, the SAUTR, the SNUS, and the CBPS. One informant also said that 
particular private companies may be of interest. 

The SHEA informants equally “naturally” mentioned the Ministry of Education, 
the HEIs, the SAUTR, and the ASHEI as central to their work with evaluation and 
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EQA policy, as these are regarded as the primary stakeholders. The ASHEI, in turn, 
had a mutual interest in the SHEA, the Ministry of Education, the HEIs, and the 
SNUS. The informant at the SAUTR stressed the relation to the SNUS, and of 
course to the HEIs, but also mentioned the Ministry of Education and the 
SHEA. Likewise, the SNUS informant talked about the link to the SAUTR but also 
mentioned other unions that organise HEI teachers, researchers, and other 
professionals with higher education degree requirements, like the Swedish 
Confederation of Professional Employees (Tjänstemännens centralorganisation, 
TCO). This informant also said that Swednet? is important for their policy work. 

The informant from the CBPS said that as they are an organisation based on 
membership organisations, their important contacts are with their members and also 
with the Ministry of Education when it comes to EQA issues in education in a 
Swedish policy context. 

When we conducted the interviews with this selection of policy brokers, it also 
became evident that they, as individuals, are themselves channels or “carriers” 
(Sahlin and Wedlin 2008, pp. 228-229) for EQA policy dissemination. As we 
showed in the chapter “National Evaluation Systems”, the number of national 
evaluations and evaluators was very restricted in the 1970s, and the knowledge they 
generated did not feed into national or institutional politics and policy in any 
substantial way; it seems as not very much brokering or dissemination was taking 
place. This has changed, but the number of actors who are now involved in policy 
brokering and act as carriers of higher education EQA policy between Europe and 
Sweden is still rather small. They move between employment at HEIs, the national 
agencies, the Ministry of Education, and other influential organisations. This means 
that the governing power within Sweden is quite concentrated to these individuals: 
they are themselves both carriers and channels. 

There are a large number of organisations, particularly European, that are 
involved in, or more specifically work with, EQA issues in higher education. 
Although the informants did not mention all possible organisations, it was 
nevertheless clear that comprehensive discussions and other types of policy work 
are broadly conducted in this respect. Dissemination of EQA policy seems to run 
through organisational links between similar actors and/or functions in the European 
and international contexts and the Swedish context. Politicians meet and disseminate 
policy to other politicians; national agency staff and quality experts meet and 
disseminate EQA policy to others in similar organisations, positions, etc. We 
interpret this to mean that the channels between European and global policy spaces 
and Sweden appear to be quite insulated from each other. But when we add the 
internal Swedish dissemination channels to the European/global channels, there is 
some evidence that EQA policy nevertheless is disseminated rather widely, all the 
way into Swedish higher education institutions, and the SHEA plays a significant 
role in this respect. (See Fig. 1) 


5 Swednet is a formal Swedish network for pedagogic development in higher education (Swednet 
n.d.). 
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Our findings show that the ENQA membership, or rather the revoked membership, 
was presented as an essential issue in the national EQA policy processes. After 
having been one of the early members, the Swedish national agency was denied 
membership in 2014. This status was a result of the 2011-2014 EQA system and 
came to be portrayed as a national and international embarrassment. We think this 
is a clear example of how the open method for coordination works. The ENQA 
membership requirements are tied to the adherence of the ESG for aspiring members, 
and the ESG are examples of: “jointly established measuring instruments (statistics, 
indicators, guidelines); benchmarking, i.e. comparison of EU countries’ performance 
and the exchange of best practices (monitored by the Commission)” (EUR-Lex 
n.d.). We have also shown that becoming a full member again was framed as an 
important political goal and used as a way to re-establish the lack of national 
legitimacy and criticism of the 2011-2014 system. The government and the national 
agency SHEA did this by stressing the European EQA policy and the ESG. As 
Sassen (2007) emphasised, this points to the importance of the policy work needed 
by the nation-state to make international/global policy attractive to the national 
context. Not only have several nation-states come to agreements over joint standards, 
but individual nation-states have had to reconcile internal conflicts and interests to 
form a receptive national context for external policy to enter and influence the 
national policy. 

Other means for European EQA policy to travel and influence national policies 
are also evident. In the case of EQA policy and practice, our study shows that 
meditative activities (Jacobsson 2010a, b) like taking part in European/international 
networks, conferences, meetings, etc. are important in governing and policy work. 
In relation to the ENQA membership situation, regulative and inquisitive activities 
also become indirectly important. This is because becoming a member means 
adhering to certain rules and regulations, ENQA statutes, and the ESG, which 
means engaging or performing activities that are in line with these requirements. 
Complying with the ESG also means opening up for reviews and assessments by 
external bodies. Taken together, these activities most likely are strong governing 
powers for those who aspire to become and remain members, as member agency 
reviews are conducted every 5 years. For Sweden, this means that when the SHEA 
applies for a new membership, such activities will take place, and the 2016 national 
EQA system was designed in line with the ENQA membership requirements, which 
we come back to in the chapter “Re-launching National Evaluation and Quality 
Assurance: Expectations and Preparations”. Knowledge carried by the policy 
brokers about how to design the Swedish EQA system to be in line with the ENQA 
requirements have been a condition for European policy to enter Sweden and hence 
influence national governing of higher education. 

This EQA policy travels well through the channels we have described and 
through the governing activities (Jacobsson 2010a, b) we discussed above. In these 
encounters (conferences, networks, etc.), knowledge about EQA is exchanged, 
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learned, and brokered. Freeman and Sturdy (2014) proposed that three forms of 
knowledge are activated in policy processes: embodied, inscribed, and enacted. We 
find that all three forms are probably present in the dissemination processes and 
channels. The policy brokers bring experiences of EQA work with them to meetings 
and conferences, which is akin to embodied knowledge that travels with “carriers” 
(Sahlin and Wedlin 2008, pp. 228-229). Inscribed knowledge is present in the ESG, 
in formal presentations, reports, and papers circulated and disseminated, and this 
kind of knowledge is easy to move and reproduce. Enacted knowledge is present in 
the actions taking place, for example, when presentations and reports are discussed 
and translated to fit into new contexts. 

The exchange of EQA knowledge is often done by comparing national models 
and systems, both entire education systems and specific EQA systems. We find that 
comparisons are important as a condition for European EQA policy influence in 
higher education in that they are necessary for making adjustments in line with the 
ESG. By exchanging knowledge and experiences of how different national (regional) 
EQA systems are designed so as to be in line with the ESG, national policy brokers 
may bring this knowledge “home” and influence the national policy context. 
Comparisons are also used as a means to know what in the dissemination of EQA 
(and other) policies is of interest, useful, and possible to get acceptance for in a 
national context. In the Swedish case, it is clear that a main policy driver for the 
development of the 2016 system was the ESG, meaning that these standards and 
guidelines became influential in the attempts to govern higher education institutions, 
a finding we will elaborate on in the chapters “Enacting a National Reform Interval 
in Times of Uncertainty: Evaluation Gluttony Among the Willing” and “Re-launching 
National Evaluation and Quality Assurance: Governing by Piloting”. 

Since 50 quality assurance agencies are full members of the ENQA and comply 
by its statutes, the EGS arguably hold a dominant position in the field of EQA policy 
in European higher education. The ESG acquire increased policy and governing 
power in this circulation of EQA ideas, as described by Sahiln and Wedlin (2008). 
The ESG are discussed, read, and worked with in relation to EQA systems designs. 
Through this kind of circulation of ideas carried by interested and influential policy 
elites, the ideas become attractive or fashionable. With reference to Czarniawska 
and Sévon (2005), Sahiln and Wedlin write about fashion: 


Fashion guides imitation and the attention of actors to specific ideas, models, and practices, 
and fashion identifies but also creates what is appropriate and desirable at a given time and 
place. This leads organizations to adopt, but also to translate, these ideas, thus changing 
both what is translated and those who translate. (Sahlin and Wedlin 2008, p. 222) 


The number of actors and people involved in EQA policymaking and dissemination 
is noteworthy. We only portray one comparatively small country. Considering the 
European and Bologna context of higher education in total makes us aware of the 
large number of people who must be involved in these policy activities and the 
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dissemination of European EQA policy to the different European Union countries 
as well as inside these countries. Lingard and Rawolle (2011) discussed this growth 
and dispersion of politics and policy in terms of a rescaling process that we find is a 
useful characteristic for EQA policy as well: 


It is the growing importance of this hybrid mix of global and national factors that we refer 
to as a rescaling of educational politics and policies, into which national policies and their 
effects are increasingly drawn and reconstituted in a global field of comparison. (Lingard 
and Rawolle 2011, p. 492) 


They also argue that: 


Another productive way of conceptualising this rescaling is in respect of the relocation of 
political authority—‘outward toward supranational entities and inward toward subnational 
groups.’ (Rosneau and Czempiel 1992, pp. 2-3, in Lingard and Rawolle, 2011, p. 490) 


In this chapter, we do not exactly point to where the political authority is located 
or if it is relocated, but clearly political authority concerning EQA in higher educa- 
tion now has a firm base within the ENQA at the European level. The location or 
relocation within Sweden has yet to be explored fully, but a tendency is re-centrali- 
sation, despite the intentions of the previously mentioned autonomy reform. We 
have noted that the ASHEI has become an increasingly powerful organisation that 
tries to bring about common policies for the HEIs in Sweden in several areas (e.g. 
resources, employment, organisation, internal governance; ASHEI n.d.) in the wake 
of the “freedoms” of the autonomy reform. This points to efforts to centralise within 
the HEI sector itself, where individual HEIs adjust to the agenda set by the 
ASHEI. An additional example of centre-local relocation may be the government’s 
recent decision to commission the SHEA to evaluate and quality assure all activities 
at the HEIs, including research (Government Office 2017). This is a new develop- 
ment that shifts authority and governing of research from the HEIs to the national 
authority (the State). 


Finally 


In pursuing the tendency to relocate authority within the higher education sector, we 
find the vice chancellors to be an important group, as they occupy a central function 
at higher education institutions concerning EQA policy and practice. In their 
capacity of being the executive leaders of the higher education institutions, they 
may be rather influential in EQA matters within their respective institutions. In the 
next chapter, their views of the mission of the university, of quality in higher 
education, and of national EQA are described and analysed. 
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in Times of Quality Assurance: gest 
The Assumptive Worlds of Vice 

Chancellors 
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Abstract In this chapter, the vice chancellors’ views of the idea of the university, 
quality in higher education, and national evaluation and quality assurance are 
described and analysed as parts of their assumptive worlds. A majority of all 
Swedish vice chancellors were interviewed, and their different and partly conflict- 
ing views are outlined as different regions of the Swedish higher education land- 
scape. As a background, the changes to the governing of higher education in Sweden 
are briefly presented. The vice chancellors’ assumptive worlds are discussed in rela- 
tion to these transformations and to governing higher education by evaluation and 
as potential grounds for their common policy and action. 


Introduction 


In the previous chapters, we delineated the background by describing the Swedish 
higher education system and the former national evaluation and quality assurance 
(EQA) systems. We further related them to a European context. In this chapter,! we 
will continue to map the scenery by focusing on the subjective understandings, or 
the “‘assumptive worlds” (Marshall et al. 1985), of Swedish vice chancellors. 

We started our project, as described in the chapter “National Evaluation Systems”, 
by gaining deeper insights into the different national EQA systems. These insights 
pointed to an expansion in the activities making up the EQA systems and to changes 
in governing by evaluations of higher education institutions (HEIs). Along with 
these changes, it was also obvious that higher education in Sweden was not isolated 
from Europe or beyond (as described in the chapter “Europe in Sweden”). The pur- 
pose, benefits, and quality of education and research at the HEIs had become matters 
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of concern for instances far away from the universities. When taking part of docu- 
ments at both the national and European levels concerning the evaluation and qual- 
ity of higher education, we found an idiom and a jargon that was hard to see through. 
The texts concern quality, higher education, and quality assurance; at the same time, 
they do not touch upon what quality in higher education is or what the idea of the 
university is and should be. These questions were never really discussed. In light of 
the changing governing of higher education, we were interested in letting the 
Swedish vice chancellors state their views on these issues. Our focus on the actors 
doing the EQA led us not only to study the work of these actors but also their ideas 
and beliefs. 

In Sweden, vice chancellors are academics with backgrounds as scholars and 
teachers within their respective discipline and within an academic tradition. Before 
1993, the HEIs’ staff elected the vice chancellor. Since 1993, the government has 
appointed vice chancellors after an internal selection process (SFS 1993:100). In 
Sweden, the vice chancellors at the universities and university colleges? are ulti- 
mately responsible for the quality of education at their HEIs. Today, they carry out 
their work in a quite different institutional setting compared to some decades ago, 
when they were able to “mind their own business” to a large degree, without much 
external interference. The questions of what a university is and what it ought to be 
are becoming increasingly politicised. In the European Union (EU), efforts have 
been made to unify the educational systems, and the far-reaching political consen- 
sus in Sweden is that it is crucial that Swedish HEIs are part of these efforts. Vice 
chancellors in Sweden thus have to consider a multitude of international and 
European as well as national and regional aspects. How they navigate in these rela- 
tively new circumstances for HEIs and how they formulate their agendas are of 
interest and relevance for their way of governing and forming policies and practices 
at their HEIs and for future joint actions within their network, the Association of 
Swedish Higher Education Institutions (ASHEI, Sveriges Universitets- och hégs- 
koleforbund, SUHF). When elaborating on decentred governance, Bevir (2011, 
p. 191) argues that “[t]he flow of politics is speech and other actions”. Actors’ con- 
tingent beliefs are meaningful parts of governing, as “there are diverse practices 
composed of multiple individuals acting on changing webs of beliefs rooted in over- 
lapping traditions” (Bevir 2013, p. 57). 

In this chapter, we will take a closer look at the beliefs and ideas about the uni- 
versity and of quality in higher education among vice chancellors of Swedish HEIs. 
Close attention will also be paid to their views on mechanisms of national EQA. The 
vice chancellors’ ideas and views will be interpreted as parts of their assumptive 
worlds — that is, as a “shared sense of what is appropriate in action, interaction, and 
choice”. Assumptive worlds, argue Marshall et al. (1985, p. 90 and 114), “provide 
the model or skeleton and connective tissue that pulls together data from various 
views of policy making”. Since the vice chancellors are the executive leaders at the 


>The difference is primarily that universities do not need to apply for degree-awarding powers to 
the same extent as university colleges (Swedish Higher Education Authority [SHEA] n.d.). 
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HEIs, we argue that studying their assumptive worlds concerning external EQA will 
give insights into the governing of higher education. 

Next, we give a short background on national EQA in Sweden, including some 
important changes in the governing of higher education. This will be followed by a 
brief account of our methodology and of the concept of assumptive worlds. We then 
present the Swedish higher education landscape and national EQA, as perceived by 
the vice chancellors. Finally, the vice chancellors’ assumptive worlds will be dis- 
cussed in relation to the transforming governing of higher education in Sweden and 
how these “shared senses” might constitute a basis for the vice chancellors’ com- 
mon policy and action. 


Background 


Ever since the signing of the Bologna agreement, there has been a strong movement 
towards comparability between, and quality assurance of, the member nations’ 
higher education systems (c.f. the chapter “Europe in Sweden”). On the European 
level, 50 agencies in 28 countries were being monitored by the European Association 
for Quality Assurance in Higher Education (ENQA) in spring 2019. On the national 
level, the higher education reform of 1993 (Government Bill 1992/1993:1) estab- 
lished a new national agency responsible for auditing the HEIs’ quality assurance 
and development programmes. In addition to this external control mechanism, the 
Higher Education Act and Higher Education Ordinance demands functional forms 
for quality assurance within all Swedish HEIs. Thus, each HEI is obliged to organ- 
ise an internal system for QA and programmes for quality development. 

The higher education reform of 1993 changed many activities in all Swedish 
HEIs. Bauer (1996) has argued that the reform clearly expressed “a shift from steer- 
ing mainly by rules and regulations, to a steering by goals and results” (p. 78). In an 
interview study of faculty and leadership from six disciplines at three Swedish uni- 
versities, Bauer (1996) found that vice chancellors and heads of university adminis- 
tration were generally receptive to the reform’s intentions to provide more freedom 
and authority to the universities, “even though several suggest that much of it is 
illusory and superficial” (Bauer 1996, p. 78). The widened autonomy was also fol- 
lowed by “increasing demands on their local statutes, more decisions, and adherent 
internal conflicts” (ibid.). 

Today, Swedish vice chancellors and HEI teachers and staff have lived with, and 
seemingly become used to, many and increasingly extensive and significant external 
and internal EQA systems for 25 years.’ Besides the 2011-2014 EQA system, the 
compulsory EQA systems as such have not been subject to protests. When the 
2011-2014 EQA system was imposed (in which the control function prevailed and 
the agency’s judgement was built on reviewing the students’ independent degree 


5As we have demonstrated in the chapter “National Evaluation Systems”, most of the national 
EQA systems evaluated the HEIs’ internal quality assurance systems (IQAs). 
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projects), many of the vice chancellors objected. The system was heavily criticised 
in the media, foremost by the vice chancellors’ network ASHEI. In the chapter 
“Hayek and the Red Tape: The Politics of Evaluation and Quality Assurance 
Reform — From Shortcut Governing to Policy Rerouting”, we give a more detailed 
analysis of the political processes leading up to the two more recent EQA systems. 

Sweden is, of course, not unique in the above changes in governance. As we have 
demonstrated elsewhere (see Segerholm et al. 2014), the transformation of higher 
education in Sweden appears to follow these international trends. Jarvis (2014) 
argues that “QA regimes are not benign managerial instruments — they must also be 
understood as part of a broader series of agendas associated with neo-liberal policy 
prescriptions that valorise market rationality” (p. 164). 

Liedman (2009) provides a historical expose of the transformation of the Swedish 
university. In a very condensed version, during the 1950s and 1960s, the collegial 
organisation received competition from bureaucratic organisation entering the 
Swedish universities, bringing with it a formalised and growing economic adminis- 
tration and several new sections, for planning and staff, for example. It also meant 
more power being transferred from the professors to the head of the departments. 
Another layer during the late 1960s and 1970s was the idea of democratisation, 
chiefly in the sense of student influence or, more accurately, representation in the 
departments’ new boards. Today, these boards have mostly been abolished, but stu- 
dents still can be represented in education programme committees and in boards at 
the faculty level. The next layer entering the HEIs in the 1980s and 1990s was a 
focus on leadership and the importance of charismatic leaders for the HEIs, facul- 
ties, and departments. From the beginning of the twenty-first century, the entrepre- 
neur made its entrance into the HEIs and contributed to a shift, where “higher 
education and research is a central part of the surrounding business world, yes 
maybe the most important engine” (Liedman 2009, p. 9). With the Bologna agree- 
ment, the ties between higher education and the business world were strengthened, 
and employability became the new buzzword. 

The transformation of higher education in Sweden has been subject to much 
debate within the academy, above all among social scientists. Conferences have 
been arranged, and books and special feature issues have been published, on the 
transformed field of HEIs (e.g. Rider et al. 2013; Ahlbäck Oberg et al. 2016; 
Karlsohn 2016; Wedlin et al. 2017). Even a novel taking place at an old university — 
with the sub-heading “When the university lost its freedom”, authored by a former 
political science professor — debates these changes (Lewin 2017). In their analysis 
of Swedish higher education policy texts from 1992 to 2007, Ljunggren and Ost 
(2008) found that the language in higher education policy has been reduced over 
time, “owing to the hegemonic tendencies of the discourse of economic globaliza- 
tion” (p. 13). Traditional academic ideals and justifications had been superseded in 
favour of this discourse and whose new key words include economic development, 
competition, mobility, comparability, and life-long learning. Hasselberg et al. 
(2013) further discusses professionalism and discretion and argues that profession- 
als today have been turned into administrators: they are expected to be “doers” 
without individual ideas on how things should be done or why. As a result, “the new 
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professionalism lacks the central ingredient of a value-based standard of good 
work” (p. 139). 

Increasing expectations from outside the academy, external funding bodies, 
employers, and partners from industry are putting pressure on academic life and 
work (Elmgren et al. 2016). Hasselberg et al. (2013) argue that the governments’ 
and regional actors’ ambition is to regulate and adapt the system of higher education 
and research to meet the demands of the global economy, of employability for the 
students, and new patents and products for the international market, “the driving 
force is the production of knowledge as an engine of economic prosperity and 
growth” (Hasselberg et al. 2013, p. 4, italics in original). In a case study of the 
Swedish Knowledge Foundation (KK-stiftelsen) and its mission to modify the new 
universities (established after 1977) into motors of regional development, Hyvönen 
(2013) found that the vice chancellors at these HEIs were enthusiastic and optimis- 
tic. For instance, the then-chief executive officer of the foundation stressed that “our 
meetings with the vice chancellors testify to the strong bonds between the vice 
chancellors and the representatives of the business community” (quoted in Hyvönen 
2013, p. 101). These bonds seemed more easily established at the new regional 
university colleges, where research funding is scarce and cooperation with local 
businesses may be appealing. 

The transformed field of higher education will most likely change the notion of 
quality as well. Hasselberg et al. (2013) predict that quality evolves into a supply 
and demand definition, i.e. “the notion of quality and demand merge” (p. 3). 

As have been shown, quality in higher education has for long been an object of 
interest yet elusive and hard to exhaustively define. Scholars have identified many 
aspects when trying to capture the essence of quality. Harvey and Green (1993), for 
example, gave an account of five basic forms, while Dahler-Larsen (2008) investi- 
gated five perspectives on quality by analysing which quality criteria, types of prob- 
lem, operationalisations, forms of normativity, and matters of avoidance they 
represent. Cole (2011) discussed educational quality in terms of research and teach- 
ing and suggested 13 quality indicators. 

In Sweden, the government’s higher education reform bill of 1993 (Government 
Bill 1992/1993:1), which had the subtitle “Freedom for quality”, was motivated by 
a need to strengthen Sweden as a “nation of knowledge” (Bauer 1996, p. 77). Bauer 
(1996) found that the university leadership often expressed quality in terms of basic, 
solid work and a conscious ambition to develop this work, as an improvement that 
must come from within the university. They problematised the balancing between 
the internal academic criteria and the external societal demands, and “[n]o one 
directly formulated quality in terms of goal- fulfilment” (Bauer 1996, p. 78). To 
ensure and enhance quality, the university leadership emphasised dialogue between 
teachers as well as between teachers and students. The teachers, on the other hand, 
demonstrated great uncertainty when asked about quality in higher education and 
also admitted that they had not thought much about it. When answering questions 
about quality, the most common answer was that quality is related to the connection 
between research and teaching. Bauer (1996) commented that quality “is often 
taken for granted in the university” (p. 79). This is hardly the case for Swedish 
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university teachers today after taking part in internal quality assurance (IQA) and 
national EQA systems since the beginning of the 1990s. 

In the next section, the method and data will be set out, and the contemporary 
Swedish higher education landscape as described by the vice chancellors’ will be 
presented. 


Methodology and Data 


Marshall (1985) underlines the potential of using interviews as a way of entering 
into the assumptive worlds of the interviewees: 


Using their utterances as a key to understanding their assumptive worlds will provide 
insights into policymakers’ worldviews, their ways of understanding and wending their 
way through their own world to achieve their own ends. (Marshall 1985, pp. 9-10) 


Following Marshall et al. (1985), who interviewed policy makers, we argue that our 
interviews with vice chancellors reflect “[h]ow these elite act, with what under- 
standing, values, and senses of what is possible and proper” (Marshall et al. 1985, 
p. 93). They have common experiences and values related to their previous aca- 
demic history and current situation. In line with Marshall et al.’s argument about 
policy makers’ assumptive worlds, we assume that vice chancellors’ perspectives 
on higher education “limit the range of options and focus debate within certain 
understood priorities” (Marshall et al. 1985, p. 110). Assumptive worlds reflect 
shared meanings that, in this case, represent the vice chancellors’ common sense 
and help them decide what is appropriate and possible and what is not. This also 
involves a common language that reflects a “taken-for-granted framework within 
which policy making occurs” (Marshall et al. 1985, p. 91). 

Also arguing for studies of meanings and beliefs, Bevir (2013, p. 70) encourages 
social science researchers to focus on “the social construction of practices through 
the ability of situated individuals to create and act on meanings”. The decentred 
approach he offers is aimed at studying social life through the meanings “that infuse 
the beliefs and practices of individual actors” (p. 70). 


It is to reveal the contingent and conflicting beliefs that inform the diverse actions that 
constitute any domain of social life. (Bevir 2013, p. 70) 


During spring and autumn 2014, we interviewed 33 vice chancellors of Swedish 
HEIs. The interview questions forming the basis for this chapter were: What is a 
university/university college? What makes it a university/university college? What 
is quality in higher education? How is it possible to reach that quality (i.e. quality as 
described by the informant)? Can evaluation and quality assurance enhance the 
quality of higher education? When asked these rather wide questions, the vice chan- 
cellors often appeared surprised and a bit disturbed or confused. Like the university 
teachers in the beginning of the 1990s (Bauer 1996), they seemed unprepared and 
not used to talk about higher education and quality in this (abstract) way. 
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The vice chancellors’ answers were analysed at three separate occasions: one for 
identifying the different ideas of the university, one for arriving at the different 
views of quality in HE, and finally one for determining their line of reasoning about 
EQA. A merger of the two first analyses is presented in this chapter, forming the 
Swedish higher education landscape, which is described as comprising four diverse 
regions. The analysis of their views on evaluation and quality assurance will then 
follow as a special aspect of this landscape. Quotations from the vice chancellors 
are used to illustrate the different regions. 


The Higher Education Landscape According to the Vice 
Chancellors 


The vice chancellors are navigating their institutions through the previously 
described changed governing of Swedish higher education. In the following, we 
present the different ideas on higher education expressed by the vice chancellors. 
We will present them metaphorically as different regions within the higher educa- 
tion landscape. We focus on the qualitative differences that constitute the contempo- 
rary higher education landscape in Sweden, rather than on giving a quantitative 
account of how many vice chancellors represent each perspective. The vice chancel- 
lors sometimes moved across this landscape and expressed more than one perspec- 
tive. Overall, the vice chancellors understood the HE landscape as fulfilling the task 
of producing new knowledge and communicating it to new generations. This was 
often the first answer to the question of what a university/university college is. 
However, when asked to elaborate on their answers, four distinct regions of this 
landscape appeared. 


The University as Freedom and Integrity 


This region accentuates the university idea as one of freedom and integrity for 
higher education institutions. 


That we can protect the freedom to be able to think freely, deciding on content and forms 
for education and research: that’s our question, and that is the important question for a 
university. (VC 3) 


It is essential that the building of new knowledge be driven by researchers and not 
by politicians or other external interests. The absence of directed research enables 
the research process to take unpredictable and interesting directions that would not 
be possible if steered from other instances. 

The freedom of research, the possibility to choose both methods and presentation and to a 


great degree objective and focus: that change, and development of new knowledge are 
driven by researchers and academics and not by political issues, for example. (VC 31) 
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The importance of critical dialogues among colleagues was emphasised, as were 
encountering and confronting knowledge. The university is a place characterised by 
“a positive, creative, knowledge-focused environment” (VC 6), where different 
views and perspectives can be brought together. “It should be a kind of melting pot’, 
meant VC 6. 

The free spirit characterising research and collegial dialogue among researchers 
also benefits the quality of education at HEIs. Quality in higher education means 
that students should be offered an education that opens up their minds and ways of 
thinking. 

...at a university, you don’t educate and teach about truths but widen the students’ perspec- 


tive and teach students to put questions to the surrounding world and also to themselves. 
That is what I think characterises a university. (VC 13) 


Teaching with quality is believed to produce education that generates and advances 
students’ ability to analyse and critically reflect on problems. These abilities should 
be nurtured during the educational programme and will be generated when acknowl- 
edged research is utilised in education, to give students opportunities to develop 
their reflective abilities by burrowing into research, trying different avenues, and 
thus critically valuing what they are experiencing: “It’s incredibly important that all 
our teachers, from lecturers to professors, are doing both research and teaching so 
that our students always get to experience the best” (VC 26). However, this kind of 
quality, with an education that is deeply rooted in lecturers’ own research, is some- 
times a problem for the university colleges. Their possibilities for securing research 
funding, and thereby having lecturers who are both researchers and teachers, are 
less than those of the universities. One of the vice chancellors from a university col- 
lege clearly stated that “the reduced funding also causes a deterioration of the pos- 
sibilities in education” (VC 31). 

By graduation, students should manifest sustainable qualities that can be used in 
future occupations and societal engagements as a whole. Stressing these more 
generic skills, such as analytical ability, might also mean that universities have to 
struggle against — and not give in to — commercial and industrial demands that 
would prefer universities to customise students according to a presently narrow 
concept: 

They [the companies] really would like to have ‘turnkey component students’ for existing 

production, students who are directly useful in production today, but I think it’s equally 


important, to say the least, that they are useful or fit the labour market in 10 or 20 years. 
Therefore, a stable ground is the absolute most important thing to give them. (VC 10) 


Evaluating the quality of higher education means providing information of the stu- 
dents’ development during the educational process, such as by reviewing if the HEIs 
“follow up on students’ abilities”, including “the ability to make critical analysis” 
(VC 3). 

In a special part of this region of freedom and integrity, quality of higher educa- 
tion meant refining the unique gifts with which students enter the education; that is, 
“they are educated towards creating something out of their own minds” (VC 32). 
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The first step of reaching quality is an accurate selection process that can succeed in picking 
the most talented students. In this selection process the initial qualities are specified as 
technical skill, musical expression, ability to analyse and so on (VC 34) 


To strengthen the students’ individual talents and visions, the education must give 
them room to develop their capacities and interests in close encounters with the 
teachers and with the institutions’ research. Students should also meet a lot of dif- 
ferent role models and have the opportunity to learn from different ways of practis- 
ing their speciality. 

Quality for us is these different aspects: high quality when they enter [the education]; very 


high competence in the teachers and material, collaboration, and media; and the surround- 
ing society, the physical nearness to professional experts in their field. (VC 34) 


It is important for an educational institution to be up to date and be “perceptive of 
innovations and support new interesting things” (VC 33). Thereby, the HEIs’ free- 
dom to choose their own lines of research is important as well. It is also a matter of 
time and resources spent tutoring the students for them to gain a certain expertise 
and confidence in their chosen field. The goal is “[t]hat you become confident and 
safe in your area, safer, so you grow in your studies” (VC 35). This self-reliance will 
be built when resources and time are spent on encouraging the students to question 
and challenge their own creativity and reflect together with supportive and more 
experienced teachers. 


The University as Societal Progress 


Close to the above region of the HE landscape characterised by freedom and integ- 
rity for HEIs, we found a second region emphasising the HEIs’ function of develop- 
ing the community and being conducive to societal progress: “The institutions have 
an important task in providing society with knowledge and also contribute to posi- 
tive change in society” (VC 29). In this region, research and tertiary education are 
emphasised as a driving force for social development and change. Such ideas criti- 
cally reflect the HEIs’ responsibility: 


This model with a core of research and education and the task of collaboration and develop- 
ment, of course it has proven historically to be a survivor (...) the characteristics are, you 
have stability, and you have the freedom to examine our contemporary society, including 
your own responsible authority, if it is a public university. At the same time, you are perhaps 
the most vital force for community development and social change. (VC 2) 


The HEIs play a key role for national progress due to their efforts to educate the next 
generation and their search for new knowledge. In this mission, it is important to 
protect “the academic integrity” (VC 17) to “make our own decisions, so this is a 
crucial thing for the university” (VC 2). This concern, as we can see, relates to the 
above presented region in its wish to protect academic integrity. This can therefore 
be said to constitute a common part of the assumptive worlds among Swedish vice 
chancellors, a part of their shared meanings or common sense. 
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This region is characterised by its emphasis on the HEIs’ contribution to society 
through research and education and thereby their commitment to the long-term mis- 
sion to change society in a positive way. 

There is no other actor in society that really can take on this responsibility of bringing 

together, taking a long-term approach forward: a responsibility the other players can’t take 


because they are more short-term oriented. Politically, or other forces, commercial forces, 
can’t take this responsibility. (VC 9) 


Through their long-term and future-oriented research, the HEIs can educate and 
prepare the next generation with the most recent knowledge. Furthermore, the HEIs 
must establish and stay in contact with other actors in society and be able to prepare 
and advance these actors’ ability to “utilize the new force that the newly graduated 
students represent” (VC 9). As such, there is ever-greater pressure on universities to 
interact with and prepare for a sustainable society as well as “to generate under- 
standing for the important challenges and linkages at hand” (VC 20). 

The quality of higher education in this region characterised by development opti- 
mism implies an institution with an education that gravitates towards preparing stu- 
dents to become overachievers, based upon international comparison. As one vice 
chancellor expressed the demands, “Well, of course it is important with a high level 
of quality internationally and that it can be compared with the best educations inter- 
nationally” (VC 8). Quality is produced and identified in collaboration and com- 
parison with high-ranking international and national universities and research 
communities. Well-functioning collaboration enables the international recruitment 
of skillful teachers, researchers, and students, and to VC 2: “there is nothing like 
having large international recruitment at your university” (VC 2). Comparison and 
collaboration with prestigious international universities are important for quality 
assurance and for setting standards for the universities’ internal quality assurance 
systems, and they also inspire and incentivise improved contributions by the staff 
and students. In addition, this international collaboration, besides benefitting a uni- 
versity’s research, education, and quality assurance, is also seen as enabling Swedish 
scholars and tutors to “influence the EU agenda for research and education” (VC 2). 
Evaluations of quality of higher education in this part of the landscape thus involves 
engaging in qualified international networks, as “they give us advice, we present our 
strategies and our thoughts to them, and they give us their views on what we should 
reflect on” (VC 2). 


The University as Regional Benefit 


A third region in the higher education landscape concerns the regional benefit and 
profitability. This region is somewhat related to the former, in the sense that the 
knowledge produced at the HEIs should benefit society. However, in contrast to the 
former region, this region is more directed towards economic and regional profit- 
ability. These institutions are important to vitalise the region: 
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University colleges in this regional perspective have an extremely important role, not least 
for the supply of competence for key functions in the region. This can be seen, for example, 
in our catchment area, where 70 per cent of our students come from this region. (VC 26) 


This benefit also seems to be mutual. Regional enterprises can be very important for 
the HEIs. Enterprises and regional driving forces thus indirectly can exercise influ- 
ence since they can contribute to smaller universities’ research and development 
funding. Hence, HEIs can receive new funding by being receptive to regional needs. 
One vice chancellor expressed this as follows: 


New universities or fairly new university colleges [like ours] are a bit closer to ‘put our ears 
to the ground’ and be part of situations where things happen. It is relatively easy to get in 
contact with us. (VC 30) 


Producing “utilizable research” (VC 1) is emphasised as being important and worth- 
while, and no contradiction seems to exist between basic and applied research; “the 
benefit extends from the ivory tower and out to the garage” (VC 25). These benefits 
may be threatened by the recent merger of smaller HEIs into bigger units, as “[t]hey 
risk losing the unique regional conditions and it gets worse for the regions” (VC 
28). 

Also in this region of the HE landscape, quality in higher education can be illus- 
trated as a form of education that conveys to the students an ability to make critical 
analyses; “these cognitive qualities that the society needs” (VC 30). The HEIs that 
succeed in doing this perform educations with quality. 


The University as a State Authority 


The final region in the Swedish higher education landscape characterises HEIs 
mainly as an official state authority. Here lies the formal and assigned commission 
that vice chancellors as representatives of an authority are obliged to perform, 
according to the Higher Education Act and Higher Education Ordinance. The HEIs 
are responsible for producing knowledge and giving students a good education. 


It is actually to have an official mission in terms of education and research, and accounting 
for knowledge production and knowledge transfer, exercised according to constitutional 
law in terms of quality, research, and education. (VC 24) 


Quality in higher education is equated with achieving (or exceeding) the Swedish 
national goals for higher education. In other words, if students taking a specific 
programme or degree reach the national goals for that programme or degree, their 
education has been of high quality. The content and competences are not specified 
for this kind of quality; rather, such goals were taken for granted. “Quality is related 
to the goals you specify”, and these goals are predefined “[a]ccording to the quality 
assurance system of the SHEA” (VC 23, both quotes). 
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As mentioned, quality in education is often taken to be of value to society. 
However, this region also has conceptions of educational quality that are as valuable 
to the individual student: 


From the point of view of the student’s period of study here, education should be as valu- 
able as possible for them from all angles. It should be an investment in their future, and 
that’s the core point in the quality concept: that we make it as good as it can ever be. That it 
is useful for the student. (VC 14) 


This type of quality in higher education can be determined through student ques- 
tionnaires and course evaluations. The results of these assessments are taken seri- 
ously: “if they answer that they have not come in contact with research, we can take 
a look at these courses” (VC 14). According to the vice chancellors, IQAs and 
national EQA systems also may identify and help improve the quality of education. 
Both are developed in agreement with the national and local goals, which therefore 
means a close follow-up on the goals. The vice chancellors often reported and elab- 
orated on ambitious IQA systems developed by their institutions to anticipate the 
national evaluation and hopefully adjust the quality of education before being evalu- 
ated by the SHEA. 

In this section, we have sketched the higher education landscape in Sweden as 
one comprising four regions. These perspectives of HEIs and quality in higher edu- 
cation constitute the vice chancellors’ assumptive worlds, in the sense that they are 
well known and openly display “what is possible and proper” (Marshall et al. 1985, 
p. 93). Even though not all of the vice chancellors sympathised with all of these 
perspectives, in contemporary Sweden, the perspectives are approved ways of stat- 
ing the idea of the university and thus to base higher education policy on. In the next 
section, we will take a closer look at the vice chancellors’ approaches to EQA. 


National Evaluation and Quality Assurance 


Can external evaluation and quality assurance promote quality in higher education? 
In all different regions of the higher education landscape, it seems, according to the 
vice chancellors, that the national EQA systems are drivers for quality improvement 
and thus part of the vice chancellors’ assumptive worlds. However, the kinds of 
quality that should be evaluated may differ. 

When the interviews were performed, the vice chancellors’ most recent experi- 
ence with external EQA was the national system operating between 2011 and 2014. 
As described in the chapter “National Evaluation Systems”, this system was heavily 
criticised by the vice chancellors’ own network, the ASHEI. The criticism foremost 
concerned the strong emphasis placed on students’ independent degree projects for 
deciding the quality of a special educational programme or discipline: “They 
measure far too narrowly (...) have to look to more breath than the degree project” 
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(VC 8). Despite this criticism, in their interview answers, the vice chancellors quite 
often suggested that this national EQA system operated to promote quality anyhow. 
One motivation behind these somewhat puzzling answers was, “When we know we 
are going to be evaluated, we shape up” (VC 25). The decisions from the agency are 
high stakes, and for the HEIs, a fail decision can mean that they will lose their right 
to award degrees for an education programme or a discipline. 

Carrying out the national EQA means initiating many activities at the HEIs. The 
self-evaluation that has been a compulsory part of all national EQA systems forces 
the HEIs to examine and describe in detail their own institution and work. Among 
other things, it makes them reflect on how their HEI is organised and engage in “the 
collegial academic dialogue about why the results are the way they are, or why it 
seems to be these effects, how can we understand this?” (VC 24). Another element 
of the EQA systems that the vice chancellors seemed to appreciate was having 
external peers looking at and commenting on their HEI: “that someone from the 
outside takes a look at the work you do is always positive. You catch sight of things 
you have forgotten” (VC 27). The vice chancellors meant that one important thing 
for an EQA system is that it is an element that sustains improvement. This statement 
is related to one of the ENQA’s reasons for excluding the 2011-2014 EQA system, 
which was that this system did not give any recommendations for improvement 
(Swedish National Agency for Higher Education 2012). Most of the vice chancel- 
lors endorsed this critique: “it was explicitly not enhancing but controlling” (VC 1). 
“It gave a very meagre material for quality enhancement” (VC 6). Instead, the vice 
chancellors wished for an evaluation and a report with recommendations that they 
could use directly for improving their internal quality assurance. This was also the 
intention in a proposition sent to the Ministry of Education from the ASHEI in 2014. 

Where the national EQA system directs its interest is of great significance, and 
the vice chancellors had proposals for other important matters to evaluate, including 
more qualitative aspects like whether the students develop their independence dur- 
ing their studies. Furthermore, concerning the EQA system’s focus, some of the 
vice chancellors had noticed that when the students’ independent degree projects 
were the focus of the national system, many HEIs invested their efforts into prepar- 
ing the students for this exercise. This can be an example of the direct governing 
effect that the EQA systems have on the HEIs — that is, what the EQA system checks 
for is also what the HEIs put their resources on. As a consequence, one of the vice 
chancellors remarked that from the perspective of the national agency responsible 
for the EQA system, it can be a good idea to shift the EQA systems’ focus once in 
a while because the institutions learn how to master and fill in templates and forms 
in order to be approved. 

Finally, a critical remark on the time and resources put on EQA was that it risks 
taking so much from the HEIs’ resources, “at the expense of other parts and [may] 
not at all work for enhancing the quality” (VC 1). 
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As we can see, the different regions of the higher education landscape in Sweden, 
as analysed from the vice chancellors’ statements, in many respects mirrors the 
historical exposé presented by Liedman (2009). The various and, to some extent, 
opposing layers representing different times in the transformation of Swedish higher 
education still seem to exist side by side among the vice chancellors. The traditional 
Humboldtian ideals of independently formulated problems for research and educa- 
tion are found in many of the described parts of the higher education landscape, 
despite their obvious challenge to parts more positive towards external cooperation, 
influence, and research funding. However, none of the presented regions are 
unknown to the vice chancellors — they are all features of the university’s history 
and of official higher education policy. Thereby, they are also parts of the vice chan- 
cellors’ assumptive worlds — that is: 


a shared sense of what is appropriate in action, interaction and choice. That sense is incul- 
cated through socialization in their distinct policy culture, and it always affects policy mak- 
ing. (Marshall et al. 1985, p. 90) 


Bevir (2013) talks about traditions in a similar manner, as “a set of theories, narra- 
tives, and associated practices that people inherit and that then forms the back- 
ground against which they hold beliefs and perform actions” (p. 50). The vice 
chancellors’ assumptive worlds can, taken together, be said to compose the histori- 
cal tradition and background of the different regions. With this shared sense or com- 
mon tradition, it is less surprising that the vice chancellors, despite the somewhat 
opposing positions in the higher education landscape, could become united in their 
critique of the 2011-2014 national EQA system. Moreover, they successfully joined 
in a collective proposition to the government, which was also published in the pub- 
lic press. The new 2016 national EQA system displayed a great resemblance to the 
ASHEI proposition. 

The part of the freedom and integrity region that identifies quality in higher edu- 
cation as the institution’s capacity to refine the unique gifts that the students bring 
with them when entering the education represents aesthetic institutions. With the 
stress on close encounters and dialogue between teachers and students, the aesthetic 
institutions seemed to be the only ones that still could uphold the traditional ideals 
of an education that closely follows and encourages each student’s development. 
None of the vice chancellors representing other HEIs expressed this view of quality 
in higher education. However, in the beginning of the1990s, the vice chancellors 
and heads of university administration still identified the dialogue among teachers 
and between teachers and students as “a necessary prerequisite for assuring and 
enhancing quality” (Bauer 1996, p. 78). So, in this respect, the vice chancellors’ 
views of how to reach quality in higher education seem to have changed since the 
1990s. 

The more traditional ideas of the university, as represented by the first two 
regions of the higher education landscape — freedom and integrity and societal prog- 
ress — both stress the importance of HEI autonomy and collegial influence in 
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directing research and education. When the future of Swedish universities was 
debated recently in a seminar arranged by Uppsala University, the participating vice 
chancellors were critical of the lack of autonomy for Swedish universities and uni- 
versity colleges. A recent study by Karran et al. (2017), comparing autonomy for 
universities in European countries utilising 37 parameters, places Sweden very low 
on autonomy, at number 22 out of 28 countries. The ASHEI, the vice chancellors 
network, initiated a discussion on a long-term perspective on autonomy through a 
new report (Classon 2018). This issue might be the next area for the ASHEI and the 
vice chancellors’? assumptive worlds to collectively play an important role in order 
to elaborate joint policies. 

In Liedmans’ (2009) historical exposé, the collegial organisation at the univer- 
sity was followed by a bureaucratic one and by a stress on leadership and on entre- 
preneurial progress. These layers can be compared with the two parts of the HE 
landscape representing regional benefit and the university as a state authority. The 
former implies that the new smaller university colleges need to be open to the inter- 
ests of regional businesses to gain research funding. Hyvönen (2013) argues that 
they are embedded in an ideology of economic growth, in which: 


commercialised research will lead to successful innovations. Innovations are good because 
they promote economic growth. Economic growth, in turn, secures welfare and general 
prosperity. Everybody wants that. Hence, everybody must work for greater economic 
growth. (Hyvönen 2013, p. 107) 


The majority of public grants end up with the larger old universities, and the smaller 
university colleges increasingly have to rely on external grants from foundations, 
local authorities, county councils, and private companies, “funders that tend to 
make far reaching demands on what research should be prioritised and how it should 
be conducted” (Hyvönen 2013, p. 107). Problems at the regional and national levels 
are increasingly viewed as something for the HEIs to deal with, with the demands 
for solutions becoming part of the definition of quality in higher education. As 
Hasselberg et al. (2013) expressed it, “the notion of quality and demand merge” 
(p. 3). 

Another of the newer regions of the HE landscape — the university as a state 
authority and, following this, the vice chancellor as an administrator of the univer- 
sity’s capacity to live up to the national goals — can be related to the new profes- 
sionalism and loss of discretion discussed by Hasselberg et al. (2013). As leaders of 
the HEIs, the vice chancellors are not expected to be led by professional values and 
ideals of good work. Instead, “professional judgement is replaced by rules, regula- 
tions, standards, management, etc.” (Hasselberg et al. 2013, p. 140). As mentioned 
earlier, when interviewed, the vice chancellors seemed a bit unprepared for the wide 
questions about what characterises a university/university college and what quality 
in higher education is. This reaction might also be related to the vice chancellors’ 
new role as administrators. When interviewed in the beginning of the 1990s (Bauer 
1996), the vice chancellors and heads of administrations did not seem to have prob- 
lems with the quality question, in contrast with the teachers at that time. 
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After about 25 years in Sweden, the permanence and repetitiveness of national 
EQA systems, which Dahler-Larsen (2011) argues is one characteristic of evalua- 
tion machines, seems to have become an obvious part of the Swedish vice chancel- 
lors’ duties and of their assumptive worlds. None of the vice chancellors maintained 
that the national EQA systems should be abolished. This can serve as a reminder of 
the Swedish history of it building its welfare state and the trust that was built among 
the citizens in relation to the state at the same time (see the chapter “Governing by 
Evaluation: Setting the Scene”; Larsson et al. 2012). The vice chancellors are part 
of this history, and their embodied knowledge (Freeman and Sturdy 2014) may hold 
traces of this trust, in turn legitimatising the national EQA systems, which are 
enacted with their cooperation at their HEIs and with other HEIs. 

The governing potential of national EQA in higher education was notably visible 
when the 2011-2014 EQA system was in operation. The vice chancellors confirmed 
this in our interviews and in interviews to evaluate this reform performed by the 
Danish Centre for Studies in Research and Research Policy (Sgrenssen and 
Mejlgaard 2014). In both interviews, the vice chancellors commented on the effects 
of adding extra efforts and resources to the independent degree project courses. This 
meant that the leadership at the university, faculty, and department levels had to gain 
insights into the state of these courses. The embodied knowledge of experienced 
teachers was requested and enacted in planning for improvements to and more 
resources for courses preparing students for the degree project. However, these redi- 
rections of resources also meant a potential loss of resources and quality in other 
courses. 

The 2016 national EQA system, with its focus on the HEIs’ internal quality 
assurance system, instead enforces the vice chancellors and the staff handling these 
systems, for them to get to know the different parts of their institution, enact this 
knowledge, and be able to describe their system to the SHEA. 


Finally 


Evaluation machines direct attention to certain criteria that “help determine what 
actors should strive to accomplish in a given activity” (Dahler-Larsen 2011, p. 206). 
As the vice chancellors testified, the 2011-2014 EQA system directed efforts and 
resources at the independent degree project courses, with a possible loss of quality 
in other courses. This illustrates what Dahler-Larsen (2012) has defined as constitu- 
tive effects. 

In this chapter, we have described the Swedish higher education landscape as one 
with inherent contradictions, older traditional academic ideals coexisting with 
newer utility, or demand-based perspectives. In the next chapter, we will take a 
closer look at how these contradicting perspectives have played out in the political 
process leading up to the latest two national EQA systems. 
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Joakim Lindgren and Linda Rönnberg 


Abstract In this chapter, we draw attention to the important political dimension of 
the governing-evaluation-knowledge nexus. The aim is to describe and analyse the 
processes leading up to the two most recent national evaluation and quality assur- 
ance (EQA) systems in operation from 2011 to 2014 and from 2016 onwards by 
analysing the formation of the respective EQA systems and the actors involved in 
these processes. The chapter outlines political justifications and ideological beliefs 
and highlights central shifts and continuities in these processes. We explore how 
formation of EQA systems can be understood within a wider context of the work of 
governing by contrasting the fast, competitive “shortcut governing” from the 2011- 
2014 EQA system with the more dialogue- and consensus-oriented process imply- 
ing a “policy rerouting” later, as manifested in the process leading up to the 2016 
EQA system. We also discuss quality assurance expansion in the context of higher 
education policymaking. 


Introdu ction 


The chapter “National Evaluation Systems” concluded that Swedish national evalu- 
ation and quality assurance (EQA) systems have varied over time. In the following, 
we take a closer look at the political processes preceding the highly debated EQA 
system in operation from 2011 to 2014 and the most recent system implemented in 
2016.' The 2011-2014 national EQA system was highly debated and made its mark 
on both the 2016 EQA system and on the processes leading up to the decision to 
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introduce this new system. As we zoom in on processes of policy formation, it will 
become clear that EQA systems do not just replace one another like the seasons of 
the year — shifts may involve deep and heated political conflicts. This chapter will 
draw attention to how EQA systems are framed by different ideological beliefs that 
are manifested in designs in terms of whether such systems ought to, for instance, 
focus on conditions, processes, and/or results. We thus want to highlight the impor- 
tant political dimension of the governing-evaluation-knowledge nexus. As Jarvis 
has observed, “Far from simply a managerial tool, the history of quality assurance 
has been inextricably political; used as much to engineer sector and organizational 
change associated with specific political agendas as it has the pursuit of excellence” 
(Jarvis 2014, p. 158). 
We ask the following questions: 


e What political justifications and framings characterise the reform processes lead- 
ing up to the respective EQA systems, and what actors were involved? 

e What continuities and/or shifts can be discerned in the respective debates? 

e How can these political debates be related to governing by evaluation? 


The rest of this chapter has the following structure: We begin by discussing our 
approach and present the data analysed in the chapter as a way to set the scene. The 
policy story we intend to tell begins with an analysis of the process leading up to the 
2011-2014 EQA system and the actors involved in this highly debated process. We 
then briefly move on to the system that was put into effect as a result of the reform 
and point to the incremental readjustments that were implemented. We go on to 
discuss the process leading up to the 2016 EQA system and highlight the overall 
bottom-up and consensus-oriented model of policymaking characterising this pol- 
icy process. The concluding discussion points to the ideological framing of quality 
assurance, issues of policy speed, and how national EQA seems to expand over 
time. 


Approaching the Politics of Evaluation in Higher Education 


The starting point for this book is the inherent political-ideological character of the 
governing-evaluation-knowledge nexus. Evaluation is an increasingly important 
activity in society as a whole, and as such, it is also embedded within a political 
frame and governing rationale (Dahler-Larsen 2011). As we noted in earlier chap- 
ters, a dimension often referred to in governance concerns centralism and decentral- 
ism. This dimension is, of course, also related to and embedded in particular national 
political histories and debates. The Swedish Social Democratic Party has made a 
distinctive mark on Swedish education policy. In the post-war era, this party has 
rarely been out of office. Parties located to the centre-right formed coalition govern- 
ments in the mid-1970s, in the early 1990s (1991-1994), and for two consecutive 
terms in the 2000s (2006-2014) (Jarl and Rönnberg 2019). The processes we anal- 
yse in this chapter thus began during the non-socialist government’s first term in 
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office in 2006 and spanned the post-2014 national election developments, when the 
social democrats formed a minority government with the Green Party (Aylott and 
Bolin 2015). 

Going back to the centralisation-decentralisation divide, social democracy has 
historically turned to centralism, the “strong state” (Lindvall and Rothstein 2006), 
and regulation as means to realise political goals, while the political right has 
emphasised decentralism and deregulation (Bennich-Bjérkman 2002). During the 
last decades, however, this polarisation has largely been dissolved as ideas on 
decentralisation, deregulation, and marketization have been embraced across the 
political spectrum from left to right, contributing to political-ideological 
convergence. 

Swedish higher education policy may not be the most politicised policy area 
within the Swedish welfare state, compared to, for instance, compulsory education 
or employment. Even so, there are important ideological lines of conflict within 
political debates concerning higher education governance and evaluation. Tangent 
to the more general political debate, there are converging ideological and party- 
political unifying standpoints in terms of common overall goals regarding, for 
instance, academic freedom, collegiality, and autonomy. However, when looking 
more closely at the political rationale on how these common and cherished values 
are to be promoted and defended, certain political and ideological differences are 
brought forward. We argue that one way to unveil and explore such lines of potential 
conflict is to analyse political debates of external evaluation systems and their 
designs. 

To explore matters of evaluation design, a general definition is a good starting 
point. Notably, conceptualisations of and in social science are seldom straightfor- 
ward, and in our case, there exist numerous definitions of evaluation. We will not 
enter that discussion here, but instead turn to Vedung, who has provided a general 
definition on this flexible concept, namely that: 


Evaluation is a careful retrospective assessment of the merit, worth and value of administra- 
tion, output and outcome of government intervention, which is intended to play a role in 
future practical situations. (Vedung 2000, p. 3) 


Evaluation may focus on different aspects of an intervention, such as a higher 
education programme or institution, for instance: 


e (Pre)-conditions/input (e.g. funds, people, instructions) 

e Processes (e.g. teaching, supervision) 

e Outcomes/results (e.g. what the students have learned; goal attainment, examina- 
tion results, assessments of student final degree projects, etc.) 


If the overall purpose is to use the evaluation for further and future improvement 
of the intervention, a holistic approach to evaluation intuitively appears valid. In 
order to understand how a certain result has come about, one must examine the 
conditions and process. It is not sufficient to only to look at the results and/or out- 
comes of the intervention; one must establish how these results came about (Weiss 
1998; Franke-Wikberg 1992). Where to put the emphasis and what approach to 
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employ in the assessment of (certain aspects of) interventions have differed. 
Different actors located in different parts and functions of the state, such as political 
parties, national agencies, HEI chancellors, researchers, etc., have displayed differ- 
ent opinions and preferences in this regard. Additionally, as the organisation and 
design of national EQA systems are ultimately and intrinsically related to state gov- 
ernance, they have been the objects of debate emanating from certain political- 
historical developments. 


Governing Evaluation: Governing Knowledge in Swedish 
Higher Education 


Early forms of evaluation were introduced as a part of the Swedish “rational social 
engineering” welfare state project originating in the 1940s (Vedung 1998; Gröjer 
2004). At that time, the expansion of the HE sector served political goals pursued by 
the Social Democratic Party in terms of social and regional justice and equality. 
During the 1950s, however, problems of low examination frequencies in higher 
education were identified, and these problems did not go away despite an increase 
in input by way of raising the number of lecturers and lecturing hours (Abdallah 
2002). In the 1960s, more elaborate pedagogical theories (e.g. frame factor theory; 
Dahll6f 1999) attempted to “open the black box” of the education process and 
explain the relative failures of the expansion of the education system in overcoming 
problems of social inequality (Abdallah 2002). These ideas were later fused into an 
evaluation model that included conditions, processes, and results (Franke-Wikberg 
1992). As we showed in the chapter “National Evaluation Systems”, national EQA 
systems in Sweden have evolved since the 1990s, and these ideas have continued to 
circulate and been put to use (albeit with a somewhat different emphasis) in compo- 
nents of different EQA systems. In this chapter, we will show that the 2011-2014 
EQA and 2016 EQA systems are no exceptions. 

While such comprehensive systems that seek to asses conditions, processes, and 
results are perceived as scientifically valid, they are also costly and have often been 
criticised for being too bureaucratic and inefficient (Grdjer 2004). As we will see in 
the following, proponents of the political right have tended to argue that thorough 
assessments of conditions and processes run the risk of interfering with the auton- 
omy and inner workings of HEIs (cf. Government Bill 2009/2010:139). Inherent in 
this criticism is a rather overlooked aspect of problems with governing by evalua- 
tion. We express this as the problem of governing situated knowledge. The basic 
idea is that attempts to assess processes of knowledge production or teaching with 
the overall purpose of improvement and/or control in fact run the risk of eroding 
important aspects of such practices. 

This critique of technological rationalism and defence of situated knowledge can 
be discussed in line with arguments from Hayek on “the role of knowledge in soci- 
ety” (Hayek 1945). Hayek argued that top-down planning runs up against a 
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“knowledge problem” that makes comprehensive, overall management of complex, 
dynamic systems inherently infeasible. According to Hayek: 


[T]hose who clamour for ‘conscious direction’ — and who cannot believe that anything 
which has evolved without design (and even without our understanding it) should remem- 
ber this: The problem is precisely how to extend the span of our utilization of resources 
beyond the span of control of any one mind; and, therefore, how to dispense with the need 
of conscious control and how to provide inducements which will make the individuals do 
the desirable things without anyone having to tell them what to do. (Hayek 1945, p. 527) 


Hayek’s argument holds implications for any attempt to govern human practices 
since the knowledge necessary for efficient resource allocation is “dispersed, sub- 
jectively held, fleeting, and largely tacit” (Foss and Klein 2013, p. 2). In this way, 
questions of governing by evaluation are not just a matter of the cost of: 


searching for, identifying, transmitting, etc. such knowledge and/or setting up complex 
mechanisms for its revelation; like Polanyi (1959), Hayek seems to have held the view that 
there is knowledge that is inherently personal and cannot be communicated at any cost. 
(Foss and Klein 2013, p. 13) 


Therefore, while knowledge ultimately resides in the heads of individuals, Hayek 
(1945) claimed that when such knowledge is somehow combined and allowed to 
evolve over time as dynamic and spontaneous systems, humans will possess knowl- 
edge that they do not develop if separated. Nevertheless — and this is important in 
relation to systems of EQA — nobody possesses all this knowledge in its totality; 
hence it cannot be codified and then collected and held by any central planner. Thus, 
it is argued, evaluation ought not focus on or interfere with the process (i.e. the 
actual work within HEIs) — this must be left to “the man on the spot” (Hayek 1945, 
p. 524), and evaluation ought only to assess the outcome of the process. 

In this chapter, we highlight how these ideas on governing evaluation are played 
out in Swedish higher education policymaking; we explore the role of knowledge in 
political justifications and framings as well as continuities and/or shifts in the con- 
temporary debates on EQA systems. We also link our exploration to issues of delib- 
eration and speed in the processes of political policymaking. 


Modes of Policymaking: Issues of Deliberation and Speed 


Evaluation systems may be organised as a top-down or bottom-up activity or try to 
combine these approaches in different ways (Franke-Wikberg 1992). The EQA sys- 
tems, the processes of designing them, and the policymaking leading up to their 
instigation can be conducted in several ways. In this context, we want to draw atten- 
tion to processes of policymaking in terms of deliberation and speed. Lewin (2002) 
argues that political decision-making can be conceived in the following two simpli- 
fied and ideal typical ways: firstly, in the form of a consensus-oriented policymak- 
ing approach, in which parties, decision makers, and other stakeholders try to agree 
as much as possible. Secondly, and in contrast, policymaking can also be 
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characterised by a competitive approach. In this mode of policymaking, the winning 
political party or parties show its/their ability to take action and not let policies be 
watered down to reach a broad consensus. The focus is rather on prioritising the task 
of getting reforms and new regulation in operation. 

On one hand, decisions made according to the cooperative model are more likely 
to be long-lasting and not to be withdrawn if a new government comes into office. 
On the other hand, cooperation and seeking consensus also entails some difficulties 
when it comes to (certain perceptions of) political accountability. Arguments have 
been raised that the competitive model can indeed be seen as more efficient and can 
result in more innovative policy solutions (Lewin 2002). Far-reaching cooperation 
and consensus-seeking across the political spectrum and among stakeholders run 
the potential risk of producing “more of the same” instead of something “new” due 
to the nature of reaching agreement and bargaining, often resulting in small, incre- 
mental adjustments (Lewin 2002). 

More recent discussions on modes of policymaking and their speed have drawn 
attention to the increasingly trans- and international character of such develop- 
ments. This is highly relevant for higher education, as Europeanisation and influ- 
ences of international organisations, such as the European Association for Quality 
Assurance in Higher Education (ENQA), are prominent in this particular policy 
field. Peck and Theodore (2015) argue that: 


The modern policymaking process may still be focused on centers of political authority, but 
networks of policy advocacy and activism now exhibit a precociously transnational reach; 
policy decisions made in one jurisdiction increasingly echo and influence those made else- 
where; and global policy “models” often exert normative power across significant distances. 
(Peck and Theodore 2015, p. 3) 


This work is informed by Jessop’s ideas on fast policy, i.e. a form of policymak- 
ing “reflected in the shortening of policy-development cycles, fast-tracking decision- 
making, rapid programme rollout (...) and relentless revision of guidelines and 
benchmarks” (Jessop 2015, p. 208). An overall starting point is that policymakers 
increasingly face temporal pressures to design and implement policy due to general 
acceleration of time and shortenings of time horizons in other social spheres of 
society (Rosa 2013). Policymaking, it is argued, thus becomes increasingly “based 
on unreliable information, insufficient consultation, [and] lack of participation”, 
often framed by “the rhetoric of crisis” or “a climate for emergency measures or 
exceptional rule” (Jessop 2015: 208). According to Jessop (2015), these conditions 
tend to privilege policy actors who are able to “operate within compressed time 
scales, narrows the range of participants in the policy process, and limits the scope 
for deliberation, consultation, and negotiation” (p. 208; cf. Takayama et al. 2017). 

We use these ideas on modes and speed in policymaking as an overall analytical 
framework to discern the policymaking styles and rationales that our analysis of the 
two national EQA policy processes brings to the foreground, bearing in mind the 
identified governing evaluation — governing knowledge issues raised in the above 
discussion. 
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In this chapter, we primarily draw on official and political documents, including 
government bills, reports, and parliamentary debates but also on different docu- 
ments produced by various stakeholders and actors in these processes. We see these 
documents as embodying “the political processes by which they are produced” 
(Freeman and Maybin 2011, pp. 164-165). Following this, we have analysed and 
collected materials produced at different stages in these political processes leading 
up to the two most recent EQA systems. This material also includes reports from the 
Swedish National Agency of Higher Education (SNAHE, in Swedish Hégskoleverket) 
and from 2013, the Swedish Higher Education Authority (SHEA, in Swedish 
Universitetskanslerscdmbetet). Reports from other stakeholder associations, for 
instance, the vice chancellors’ organisation, the Association of Swedish Higher 
Education (ASHEI, in Swedish SUHF), the university teacher union SULF, and the 
student association SFS, are included. The process and the resulting national EQA 
system have been exposed in several media outlets and form a background for our 
understanding of the debates. We also draw on data from interviews with a number 
of actors involved in these processes. In particular, we turn to the project’s inter- 
views with policy brokers (PB), including individuals who were actively involved in 
these processes. 

We did a qualitative-directed content analysis (Hsieh and Shannon 2005) of the 
text and transcripts in which we initially identified relevant passages of text and then 
examined these passages in more detail by paying special attention to the argu- 
ments, justifications, and attempts of legitimation embedded in the processes lead- 
ing up to the respective EQA systems. We finally add that we do not intend to 
elaborate on all details in these processes but rather to highlight some important and 
signifying traits and moments. 


The Process Leading up to the 2011-2014 EQA System: 
Shortcut Governing 


Through a government decision in 2008, the evaluation system for 2007—2012 was 
cancelled due to heavy criticism from the higher education sector. The argument 
was, briefly put, that there was too much focus on conditions and too little on pro- 
cesses and outcomes. One of our informants, a key actor involved in the design of 
the system that was supposed to replace the revoked system, recalls the overall cri- 
tique and debate and that the Director General at the SNAHE used to banter about 
the bureaucratic meticulousness of the 2007-2012 system: “It was all about count- 
ing the number of senior lecturers”, but “what these lecturers were actually doing — 
if they were teaching at all — was not checked” (PB 7). Critics argued that the system 
was not transparent and thus became out of touch with reality, and it demanded that 
HEIs fill in “tidy Excel sheets that required information ‘not of this world’” (PB 7). 
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In 2009, the Ministry of Education formed a group led by a university vice chancel- 
lor that included representatives from the SNAHE, ASHEI, and the main student 
union. This group delivered a report with suggestions on how to design a new 
national EQA system. According to another of our informants, who had been work- 
ing within the SNAHE as an architect of the 2007—2012 system, the ASHEI slowly 
became a “power broker” within the HEI sector during this time: 


Initially, [ASHEI] was an interest group without any formal status. It was a small club for 
the vice chancellors where they met and discussed common affairs and where they 
exchanged experiences, but during this time ASHEI developed into a very potent power- 
broker. There were many reasons for this (...) The new [SNAHE] Director General, for 
example, did not understand that he did no longer belong to the circuit around ASHEI, 
where he had been an influential person. (PB 2) 


This informant described a cultural shift in which boundaries between the juris- 
dictions and functions of assessors and those assessed were blurred: 


You cannot let those who are to be assessed set the conditions for the assessment, but of 
course, there must be dialogue and mutual understanding. There must be mutual respect 
among the parties and for the different functions of those parties. (PB 2) 


At this point, new forms of interchange between actors within the agency and 
HEIs were initiated. According to Policy broker 7, this was a strategically important 
transformation: 


It was a cultural clash, because N.N. and N.N. [authors’ remark: names omitted] and many 
of their disciples shared the conviction that the agency must not involve the sector too much 
when designing a system. (...) We had the idea that when you design a new system, you 
darn [well] must involve the sector a lot, you must build it with the sector (...) [and for] 
these systems to work, you must work together with the sector and design the systems so 
that the people within academia find them relevant. It was a pity that, in a sense, we did not 
get the chance to develop our system further, because it was [an open] window that I think 
will never be open again. (PB 7) 


As we will see later in this chapter, this window of dialogue was soon slammed, 
but eventually it was opened again as ideas on trust, cooperation, and engagement 
were reintroduced. 


A Rejected Agency Proposal Caused a Stir 


As mentioned, the Ministry of Education gave an assignment to the SNAHE in 
2009. The agency was to develop a design for a new EQA system to include so- 
called quality evaluations that would highlight and measure results. Weekly meet- 
ings were held with the ministry and agency, and the SUHF and SFS were also 
partners in the discussions. About 6 months later, the work resulted in a report to the 
ministry emphasising three aspects in particular: expected learning outcomes and 
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examination, achieved learning outcomes, and students’ experiences and influence 
(SNAHE 2009). According to Policy broker 7, it was a pared-down system that 
deliberately broke with the previous system. It was deliberately “very, very 
slimmed”. According to this informant, there is a risk for evaluative systems to 
expand if experts are allowed to design them. We can see that discussions on evalu- 
ation design are not only about the individual parts of systems but about the size and 
scope of systems: 


You cannot let those “quality people” draw up [these systems], because it’s like in every 
other area of expertise: They want to do everything. The previous system was enormous. I 
think that it included five components; it was way too bulky. (PB 7) 


The ministry then worked on a government bill (2009/2010:139) that was to be 
presented to parliament. As this work progressed, it became clear that the Ministry 
was not completely satisfied with the SNAHE report and wanted to make further 
elaborations and amendments. The Ministry, however, did not involve the SNAHE 
in these further discussions, and the SNAHE report was turned down. It was accused 
of not focusing enough on outcomes and thus did not respond to the terms of refer- 
ence the working group had received. 

According to Policy broker 3, who at the time was a key political actor within the 
centre-right government, the argument for turning down the report was based on a 
basic “ideological view” of autonomy that was congruent with a particular evalua- 
tion design. In the words of Policy broker 3: 


It [the Ministry’s argument, authors’ remark] was based on an insight that HEIs have been 
micromanaged in many ways. (...) So, the idea was to decrease micromanagement based on 
the insight that it is not efficient. It is not efficient to seek to govern complicated activities 
like higher education and research through micromanaged organisations (...) The debate 
was greatly about [whether] “‘it [is] right to have a system that concentrates on the results of 
activities” or a system where evaluation is about [determining whether] “the processes [are] 
found to be good”. I think the latter option is wrong since you end up inside the activities 
fiddling around with how things are done instead of looking at the results. (PB 3) 


One expression of this ideological conviction was the attitude towards the 
planned use of “learning outcomes” within the evaluation design in the rejected 
SNAHE report. Policy broker 7 asserted that these “learning outcomes and the 
assessment of whether these outcomes were attained became the ‘hate object’ 
[authors’ remark: a strong Swedish word to indicate what attracts discontent] of the 
ministry and a reason for why the report was turned down” (PB 7). 

Eventually, the Ministry of Education presented their version of a new EQA 
system. This was done with the bill “Focus on knowledge — quality in higher educa- 
tion” (Government Bill 2009/2010:139). Before this bill was submitted to the par- 
liament, actors such as the ASHEI and the SNAHE had tried to argue in favour of 
the old rejected report, which was claimed to have been largely endorsed and 
approved by the HEI sector, in contrast to the hastily prepared bill (cf. Kaliber 
2013). In the bill, the government stated that the system delivered in the SNAHE 
report was not in line with the government’s ideas on autonomy, as it focused too 
much on expected learning outcomes in curricula and forms of examinations, i.e. 
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issues related to planning and realisation of teaching and learning activities. It was 
concluded that: 


Experiences from previous evaluation systems show that evaluations are powerful instru- 
ments of governance. Grounds of judgement that target how the internal processes are 
designed in order to reach the goals risk producing unintended steering effects, hence such 
grounds of judgement ought not to be part of the national evaluation system (...) The 
assessment ought to primarily concern actual education results and not be based on docu- 
ments that merely display the intentions of a HEI when it comes to carrying out education 
with good results. (Government Bill 2009/2010:139, p. 13) 


A Heated Political Debate 


In June 2010, a parliamentary debate was held (Parliament Minutes 2010). The 
political opposition argued against both the design of the system and the “shortcut” 
policy process behind it. The new proposed national EQA system was criticised for 
its narrow focus on results and for being micromanaged by the political (Government 
Ministry) level and thereby not allowing HEIs sufficient autonomy. In the debate, 
the social democrat Mikael Damberg, who represented the political opposition, 
summarised the critique. He said: 


The government has — on its own and totally unprovoked — bulldozed over the entire higher 
education sector and in addition, on its own — as it has been said — “played around as if they 
were a national agency for higher education” by knocking together a homemade micro- 
managed quality evaluation system. This is unbelievable. Because the preconditions have 
never been better than they are today: There is a deafening unity in the Swedish Parliament 
[concerning a need for a new system]. (...) This should have been done in close cooperation 
with the HEIs, ASHEI, and the students. But when the work was done and anchored, the 
government jumped in like an elephant in a glass shop and messed around. They came up 
with a new system that was presented through a couple of slides in an oral presentation at a 
meeting with the HE sector just 20 days before the bill was sent to the parliament. 
(Parliament Minutes 2010, Mikael Damberg, Social Democratic Party) 


The quotation points to a critique aimed at the democratic process itself. Thus, 
accusations of micromanagement that the centre-right wing directed to system 
designs under social democratic rule (including input, process, and results) are now 
directed at how such systems are politically processed and decided (top-down with- 
out deliberation). Minister for Higher Education and Research Tobias Krantz replied 
in the debate and underscored that the nature of the political process and policymak- 
ing is in fact about conflicting perspectives and eventually also about determination 
and the ability to act in times of urgency: 


The government bill has caused debate. This is excellent. It is good that important bills are 
debated and that they excite people. If the Swedish higher education sector was quiet we 
would have a reason to be worried. There are voices in favour of the bill, and there are oth- 
ers against it. The best way to avoid criticism is to do nothing. However, Sweden is facing 
a very important and big challenge. This is not the time to sit with our arms crossed. Sweden 
shall have a world-class higher education. (Parliament Minutes 2010, Tobias Krantz, 
Liberal Party) 
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Policy broker 7 also underlined the virtues of a decisive and efficient mode of 
policymaking in that even though deliberation is crucial, fast policy is needed to 
produce change. This informant argued in favour of line management and against 
traditional collegiality as a solution to this problem: 


If the ambition is to be at the front of the development in an international perspective, then 
HEIs cannot only be managed collegially, because not much change will come about. (...) 
We were convinced that we [actually] needed something [other] than what we had. (PB 7) 


The discussions above about the political process leading up to the 2011-2014 
EQA system reveal different ideological approaches to HE governance and evalua- 
tion of HEIs. These ideas are closely related to essential overall goals of and in 
higher education, such as academic freedom and social and economic development. 
These ideas were fused into and manifested in concepts such as autonomy and col- 
legiality and also, as we have seen, different designs of EQA systems. At the end of 
the day, however, actors across the political spectrum appeared to be willing to 
govern HEIs through evaluation with somewhat different technologies. 


Implementing the Debated EAQ System 


Somewhat paradoxically, and in contrast to the government rhetoric, a parliament- 
appointed evaluation carried out by a group of Danish evaluators showed that the 
results of the 2011-2014 EQA system did seem to govern internal processes within 
HEIs in profound ways (Sørensen et al. 2015). The increasing orientation of actors 
within HEIs towards expected learning outcomes implied a thorough implementa- 
tion of the Bologna reform. This was manifested through careful planning, assess- 
ment, and documentation, which was put forward as one of the most important 
success factors in this EQA system (Sørensen et al. 2015, cf. SHEA 2015, 2017). In 
a likewise paradoxical manner, the 2011—2014 system had a centralising effect and 
produced a continuous expansion of quality assurance work. One vice chancellor 
who was interviewed in the evaluation described the changes imposed by the EQA 
system in the following way: 


Previously, we were incredibly decentralised, and in principle, all quality work was man- 
aged locally in the organisation. But we have centralised it, because it was needed (...) This 
evaluation model was very non-transparent to many actors within the organisation (...) 
After the first round [of evaluations] we could see that this was not going to work, so what 
we did was to set up acentral... firstly, a central support function, but also, really, an organ- 
isation where the areas, that is our faculties, have their organisational representatives 
involved who can support and assist. (vice chancellor in Sørensen and Mejlgaard 2014, 
p. 22) 


This tendency towards expansion and centralisation will be further elaborated in 
this chapter and later in this volume. Now we turn to the process initiated during the 
implementation of the 2011-2014 system that resulted in the 2016 EQA system. 
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The Process Leading up to 2016 EQA System: Policy 
Rerouting 


In the following, we will show that how EQA systems were politically “done” 
changed as ideas on dialogue and mutuality gained ground as a response to the pre- 
vious perceived shortcut process. This dialogue was embedded within a discursive 
frame that, at least on the surface, appeared to slow down the speed of policy. As we 
shall see, deliberation and dialogue involve time-consuming work by actors. In 
addition, ambitious evaluation designs that result from the cooperation and 
consensus-seeking process may in turn produce new undertakings within organisa- 
tions as national EQA expand. Slow policy may thus become a functional condition 
for social acceleration (Rosa 2013). 

In April 2014, just a few months after the Swedish agency lost its membership in 
the ENQA, SHEA University Chancellor Harriet Wallberg was commissioned to 
propose a new quality assurance system for the higher education sector by the same 
Government that did the shortcut and introduced the 2011-2014 EQA system. This 
meant rerouting the process and the EQA system. Looking at the process, concepts 
like mutual trust, collaboration, and dialogue were emphasised, and the work was 
done in collaboration with different reference groups, including HEI, student, 
agency, and working-life (including unions) representatives. This involved efforts 
and work from several actors. There were meetings with HEI representatives; min- 
utes were written and stored at the ministry (Ministry of Education 2015), and a 
consensus-oriented approach was stressed (cf. SHEA 2014a). Of course, such meet- 
ings and procedures had been taking place in the previous process leading up to the 
2011-2014 EQA system as well, but as we will exemplify, the narrative to be con- 
veyed in the new process was to explicitly declare the importance of coordination, 
cooperation, and seeking dialogue. In this context, trust comes across as a central 
notion in this discursive construction of values and meanings. 


Trust of and in Swedish Higher Education and Beyond 


In our interviews, many key policy brokers expressed hopes for the future and that 
such deliberations would result in a system based on trust: 


The big differences between EQA systems are between systems that trust HEIs and systems 
that do not trust HEIs. Sweden has had a system that did not have any trust in the HEIs. We 
have had that for many, many years in different forms. I hope that we are moving onto a 
system that puts trust in the HEIs. (...) Ultimately, the political level must decide if it can 
put trust in the hands of the higher education sector. (PB 4) 


As demonstrated in the chapter “Europe in Sweden”, policy brokers also adopt 
and circulate ideas on national and international arenas, carrying knowledge from 
international to national discussions. One of our informants who was involved in 
such international collaborations and exchanges compared the European 
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development with the situation in Sweden. This policy broker highlighted an 
encounter with EQA systems within a wider European meeting and setting also 
relating to the issue of trust: 


You could feel, not just the systems on paper, but people of flesh and blood, talk about their 
experiences. One important conclusion that we brought [home] with us was that the 
Swedish culture on [the] evaluation of higher education specifically, and perhaps more 
generally, is characterised by far less trust than what we could see in these countries (...) 
the autonomy of HEIs had a completely different and profound meaning abroad. People 
within the HEIs took on the responsibility, and within the equivalent to SHEA and the 
Ministry of Education, there was respect and trust in that HEIs actually took on this respon- 
sibility. So right now, we are hoping that all the work with the proposal [the forthcoming 
2016 EQA system, authors’ remark] (...) will lead not only to a new improved EQA system 
but [will] promote a certain an academic culture in Sweden. (PB 1) 


This quotation is also significant in that it highlights that Europe (and the ENQA) 
became a more dominant point of reference in the preparatory work in this policy 
process compared to the 2011-2014 process. Importantly enough, the SHEA 
Director General was quoted on the SHEA English website: “When implementing 
the new system for quality assurance for Swedish Higher Education, the European 
Standards and Guidelines must be taken into account” (SHEA 2014b) — thereby 
strongly conveying the message to the English-speaking SHEA website readers that 
Sweden and the agency are committed to ensuring that the new EQA system will 
meet international and European standards. It can be noted that the same text was 
also published on the ENQA website under the “News” section (ENQA 2014). 

Later, on the Swedish SHEA website, the Head of the Department of Quality 
Assurance at SHEA described the process: 


It was important for us to invite a dialogue before we have a complete decision basis. Also 
[the HEIs, authors’ remark] will be using the new system, since it is important they are 
involved (...) After having discussed the proposal for a new national quality system with 
over 400 people, we are bound to say that the response has been largely positive (...) We 
have received important input (...) both before we took the final decision and before the 
pilots that are starting this autumn (...) Most important is perhaps that we felt that there was 
trust [authors’ remark: among people in the higher education-sector] in the new quality 
assurance system. (SHEA 2016) 


Shifts and Continuities in Governing (by) Evaluation? 


In contrast to the former process, the 2016 EQA system proposal was not only 
deliberated as it was developed and designed; it was also formally sent to stakehold- 
ers for referral. In the 2016 EQA process, the SHEA was assigned responsibility for 
further elaborating on how to design the 2016 EQA system (Ministry of Education 
2015). In contrast to the previous process, the SHEA was thereby granted an impor- 
tant role as an expert agency in the process of designing and implementing the 2016 
EQA system. The rerouting thus not only targeted the process but also the content 
and activities in the EQA system. This rerouting may result in a different mode of 


96 J. Lindgren and L. Rönnberg 


governing by evaluation, and in the forthcoming chapters in this volume, we will 
address this potential shift in more empirical detail. In certain ways, and on the basis 
of the data presented in this chapter, it is possible to conclude that the new EQA 
system implied a form of shift even before its implementation. This shift refers to 
the values used to frame the processes leading up to the respective EQA systems as 
“shortcut governing” to be “rerouted”. In addition, the 2016 EQA system implied 
some partial continuities as well as changes — both of which were embedded within 
the common discursive framework of trust and mutuality. The 2016 EQA system 
would not primarily be an assessment of students’ independent projects (final degree 
projects), indicating a break with the former 2011-2014 EQA system. A much high- 
lighted “new” (but essentially recycled; see chapter “National Evaluation Systems”) 
dimension of the 2016 EQA system was the SHEA assessment of HEIs’ internal 
quality assurance work. But some activities from the former EQA systems remained, 
even if reformed — for instance, thematic evaluations and evaluations of education 
courses and programmes (SHEA 2014a). Such shifts and continuities in processes 
and EQA systems also inform our concluding discussion. 


The Politics of Evaluation and Quality Assurance Reform 


In this chapter, we have described and analysed the processes leading up to the two 
most recent national EQA systems in operation from 2010 to 2014 and from 2016 
and onwards. There have been heated political and ideological debates at times, par- 
ticularly during the first period, inflicted by personal interests and conflicts, involving 
both high-level official resignations and public disputes (cf. Kaliber 2013). 


It Is Natural to Measure and Assure Quality in Higher 
Education 


Despite turbulence and change, a fundamental and important continuity character- 
ises both periods and systems. For example, no substantial public objection or strug- 
gle existed about the actual need for the state, via its agencies, to (continue to) 
evaluate, assure, measure, and steer HEIs. Hence, there is a strong cross-party con- 
sensus about the need for such policies and activities in both processes. A Social 
Democratic MP summarised the common and widespread political assumption 
about the need for external evaluation in the following statement from the 2010 
parliamentary debate: 


It is natural to measure and assure quality in higher education. The state finances higher 
education and needs to know if tax money is used the best way possible (...) Students have 
a right to know the quality of the education they are attending (...) [and] they also have the 
right to know how useful their education is on the labour market. (Parliament Minutes 
2010) 
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Nevertheless, as we have seen, the ideological framing of autonomy and/or trust, 
academic freedom, etc. has been quite different within the two modes of policymak- 
ing in the analysed processes. In this way, the chapter has shown that stakeholders 
represent issues related to the politics of evaluation and quality assurance reform in 
partly different and conflicting ways within and across the two decision-making 
processes. As was also made evident, these problematisations are all directed 
towards justifying how to design the (perceived best) external national evaluation 
system. In this way, the inherent taken-for-grantedness that a desire indeed exists for 
such a system to be implemented in the first place goes largely unaddressed. 

The act of extensive national external assessment of and in HEIs is not only taken 
for granted but also naturalised as a taken-for-granted part of the mindset of each 
actor and stakeholder in the sector. To illustrate this far-reaching striving for 
enhancement (cf. Saunders 2014; Stensaker 2007), a Social Democratic MP made a 
public and almost religious pledge to all higher education stakeholders, arguing that 
quality development is an approach to life: “Every morning when you wake up, you 
shall repeat a mantra about how you can work with continuous quality improve- 
ment” (Parliament Minutes 2016). 

Yet, even if all political actors subscribe to traditional academic values, they 
also — albeit with different ideological underpinnings — to some extent show con- 
flicting ideas on evaluation designs. Still, they unanimously push for quality assur- 
ance reform as well as to the need to measure. These simultaneous and somewhat 
contradictory stances are not easily aligned. As noted by Jarvis (2014), HEIs are 
caught between two “narratives” that appear to be difficult to interweave: 


The university sits oddly amid two narratives; one that prizes academic freedom, indepen- 
dence of thought and expression, heterodoxy and exploration to create new knowledge 
frontiers, on the other hand, an increasingly intrusive series of regulatory regimes that seek 
to manage, steer and control the sector in ways that serve the interests of the state and the 
economy by applying specific ideational motifs about efficiency, value, performance, and 
thus the economic worth of the university to the economy. (Jarvis 2014, p. 156) 


A more easily managed interlinkage is that the new government used the shortcut 
policymaking narrative politically to justify the rerouting of the latter process. In the 
process leading up to the 2011-2014 EQA system, academic freedom was framed 
as a value that is protected and defended, whereas the system was designed to only 
assess the outcomes, in the debated assessment of students’ degree projects. This 
EQA system was said not to assess and thereby interfere with the (HEI internal) 
processes and preconditions leading up to the educational outcomes (Government 
Bill 2009/2010:139; cf. Parliament Standing Committee on Education 2010). 
However, when looking at the implementation, this EQA system did not produce a 
tangible rise in HEI autonomy, and evaluations have pointed to the further centrali- 
sation of quality assurance work within the HEIs (Sørensen et al. 2015). This EQA 
framework produced a more thorough implementation and enforcement of interna- 
tional and national rules and policies as well as a common set of learning outcomes 
that became a cohesive and obligatory reference for planning, teaching, and exami- 
nation in HEIs. In this sense, it appears to have implied less autonomy, rather than 
the opposite (see also Toots and Kalev 2016). 


98 J. Lindgren and L. Rönnberg 


We wanted to place the political processes leading up to the national 2011-2014 
and the 2016 EQA systems in the limelight. In this chapter, we sought to show the 
attempted framings by the actors involved and concluded that a shortcut and fast- 
paced (Jessop 2015) policy style was replaced by a more consensus- and dialogue- 
oriented approach in the latter process (cf. Lewin 2002). In this vein, a relocation of 
the framing of and within the political discourses leading up to the 2016 EQA sys- 
tem also seemed to occur. We claim that certain non-government organisations, 
such as the nationally powerful ASHEI and, later on, the ENQA, came to set impor- 
tant and distinctive marks on the policy processes and contributed to the shift that 
ultimately came to position the 2016 EQA system in a discursively different way. 


Expansion and Complexity when Governing Situated Knowledge 


Significant in this development is also that the expansion of EQA-ambitions was 
done by bringing the HEI actors back in, by offering deliberations and giving HEI 
representatives a place at the drawing board. Such involvement did not result in less 
evaluation or auditing. Somewhat paradoxically, the HEI deliberations in the second 
process did not “roll back” state attempts of governing by evaluation. Rather, the 
earlier observation by one of our interviewed key policy actors draws attention to a 
plausible explanation: if the so-called quality people, i.e. those with expertise in 
quality assurance and evaluation, are involved in constructing evaluation systems, 
“they want to do everything” (PB 7). Thus, this chapter has illustrated the ongoing 
expansion and the complexity of quality assurance — as well as the importance of 
“expertise” and thereby of knowledge of and within these processes. Moreover, we 
have shown how the emerging cadre of professionals that we refer to as “qualocrats” 
mobilised themselves as power brokers by tearing down the previous boundaries 
between the subjects and objects of evaluation. This observation is also helpful to 
understand issues of resistance, more precisely the lack of critical debate concern- 
ing national EQA systems. This is a topic that we will return to later in the book. 
We find that the analysed Swedish higher education case illustrates how “slow 
policy’, in the form of dialogue and deliberation, is contributing to increased com- 
plexity and becoming a functional condition for social acceleration (Rosa 2013). In 
the beginning of this chapter, we also discussed the wicked problem of governing 
situated knowledge and turned to Hayek (1945) for illustrations of the risk that 
externally assessing processes of locally held and embodied knowledge production 
or acquisition will erode important aspects of such practices. Even if the data anal- 
ysed in this chapter cannot provide a clear-cut answer on handling this problem, the 
data indeed point to the political attempts and politically articulated will to (a) 
assess and thereby (b) govern such processes, albeit by different means and tech- 
nologies. Such manifestations are the different modes of policymaking and the 
associated paces for the work of doing governing they entail. Overall, however, we 
suggest that ideas and reforms on autonomy launched by the centre-right govern- 
ment did not manage to unfetter higher education actors from external accountabil- 
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ity pressures. On the contrary, these policy attempts appear to provide an example 
of what Graeber (2015) terms “the Iron Law of Liberalism”, which states that: 


any market reform, any government initiative intended to reduce red tape and promote 
market forces will have the ultimate effect of increasing the number of regulations, the total 
amount of paperwork, and the total number of bureaucrats the government employs. 
(Graeber 2015, p. 9) 


Finally 


In the upcoming Chaps. 7, 8 and 9, we will unpack these developments in more 
detail as we empirically study the designs and local enactments of evaluation in 
HE. We will also show how the evolving group of professionals working with qual- 
ity issues in the higher-education sector, the qualocrats, are making their mark in 
these processes of quality assurance enactment. Questions of autonomy, power, and 
expansion have been accentuated over time, and we will continue to explore how 
these issues unfold as the 2016 system is designed and put in place. The next chap- 
ter, however, focuses on how the debated 2011-2014 EQA system was carried out 
in terms of its media display and framing of evaluation results from evaluations of 
study programmes within individual HEIs. The next chapter thereby elaborates on 
the mediatised activities and work undertaken within the 2011-2014 EQA system 
and draws attention to how the communications of and with the media were formed 
and framed by both the responsible national agencies and the HEIs themselves. 
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Abstract The aim of this chapter is to analyse how results from the 2011 to 2014 
national evaluation and quality assurance (EQA) system were communicated to and 
via the media. First, the analysis focuses on the attempted media framing, as mani- 
fested in press releases of national evaluations from the responsible national agen- 
cies. Second, the higher education institution-media interactions, in the context of 
two national quality evaluations from two subject areas (education and specialist 
nursing), are analysed from the perspective of how four higher education institu- 
tions’ attempted framings were (re)presented by the media. The chapter concludes 
with a discussion pointing to interdependence and possible reinforcement of the 
media-quality assurance relationship and points to some possible implications for 
education governing. The chapter also discusses the silences and articulations that 
could not be detected in the studied data, as situated within the context of reputation 
management and media display in the contemporary “evaluation society”. 


Introduction 


This chapter continues to explore the governing-evaluation-knowledge interactions 
in Swedish higher education, now in the context of media exposure, display, and 
management. This chapter analyses the highly debated national quality assurance 
and evaluation (EQA) system in operation from 2011 to 2014 that was described in 
the previous chapter, with a focus on how the actual evaluations performed as a 
result of this EQA system were mediated and displayed “in the public eye”. Today, 
media reporting on national evaluations and quality assurance are high stakes for 
the parties involved. For the higher education institutions (HEIs), attracting future 
students and a favourable “branding” not only depends on the actual outcome of the 
evaluation but also, importantly enough, on how the outcomes are reported and 
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represented by the media and thereby transmitted to the wider public and to differ- 
ent stakeholders. Media display is also important for the evaluation agencies, and 
media coverage is in itself displayed as a sign of both policy and agency 
“success”. 

As described in the chapter “National Evaluation Systems”, this has not always 
been the case. Since the introduction of the per-student state grant system, the incen- 
tives for HEI branding and PR activities as well as the potential risk of unfavourable 
media exposure have risen considerably. In this context, the 2011-2014 EQA sys- 
tem can be said to have raised the stakes even more. It included both sanctions and 
rewards. Each HEI and programme was given a grade. The top performers received 
extra funding, and the lowest grade could result in a revoked licence to issue degrees. 
In addition, the evaluations of study programmes and the resulting grades were to 
be comparable across universities, as a form of “customer information” for prospec- 
tive students. To aid in this ambition, the evaluation reports were standardised, made 
publicly available, and intended to speak to a wide audience. Taken together, the 
design of the 2011-2014 EQA system seems to go well with the “media logic” (see 
below), in which winners and losers, the potential for scandal, and the issuing of 
rewards and sanctions tend to be prioritised angles of reporting. 

The aim of this chapter is to analyse how evaluation results were communicated 
to and via the media, by studying media communication and the display of national 
EQA, in the form of quality evaluations of higher education study programmes in 
operation from 2011 to 2014. The following questions are addressed: 


e How were the evaluations communicated to the media, and how did the regional 
media portray the HEIs in the context of national quality evaluations? 

e Did the media coverage reinforce the representations attempted by the responsi- 
ble agency and the HEIs, or were these images challenged? 

e How can the media-quality assurance relationships be understood in light of the 
two questions above, and what are the implications for governing? 


Next, some conceptual tools and frames are introduced along with some brief 
notes on the empirical data used in the particular study reported in this chapter. This 
is followed, first, by a mapping of the attempted framing of the evaluation results 
made by the two national evaluation agencies (the Swedish Higher Education 
Authority and the National Agency for Higher Education, with the first replacing 
the second in 2013), by analysing their press releases from 2011 to 2016. Second, 
two particular quality evaluations are focused upon, namely, education and special- 
ist nursing, to highlight how four HEIs’ attempted framings were (re)presented by 
the media. The chapter concludes with a discussion of these findings by pointing to 
the interdependence and possible reinforcement of the media-quality assurance 
relationship and by arguing that these have important implications for education 
governing. 
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Approaching the Media-Quality Assurance Relationship 


This chapter focuses on some of the intersections of the “media society” and the 
“audit society” in the form of public agency evaluations of HEIs and seeks to 
explore some of the dual dependencies that these relationships entail and fortify, 
arguing that they constitute interesting tensions for further exploration from the 
perspective of governing. On the one hand, the media provide certain interpretative 
frames by conditioning “rules of the game,” by allowing certain voices to speak and 
silencing others, and by operating according to a particular format and logic of com- 
munication (Hjarvard 2013). At the same time, media reporting depends on receiv- 
ing certain information and material from perceived credible and legitimate sources, 
such as agencies and universities (cf. Fredriksson and Pallas 2016; Thorbjgrnsrud 
2015; Rönnberg et al. 2013). 

The study reported in this chapter draws on literature related to different dimen- 
sions of media-education governing interactions, including (a) governance and gov- 
erning work (Clarke 2015; Bell et al. 2010; Newman and Clarke 2009); (b) literature 
conceptualising the relationship between media, society, and policy/politics/bureau- 
cracy (Christensen and Gornitzka 2018; Crow and Lawlor 2016; Thorbjgrnsrud 
2015; Hjarvard 2013; Strömbäck 2008); and (c) literature more specifically target- 
ing the media in the context of education as a policy field (Rawolle 2010; Thomas 
2009; Anderson 2007; Gewirtz et al. 2004). 

In the contemporary “audit society” (Power 1999), scrutiny, evaluation, and con- 
trol are prominent means of governing institutions, organisations, and professionals 
(Dahler-Larsen 2011; Grek and Lindgren 2015). The field of higher education is no 
exception. Under the umbrella of New Public Management, different evaluative 
activities are linked to and promoted by developments entailing the increased mar- 
ketisation and privatisation of public welfare, in both Europe and beyond. Within 
such an agenda, higher education is increasingly conceived as a form of private 
good (Englund 1996), which positions students as consumers and quality evalua- 
tions as means by which to assist, account, regulate, and even fortify these relation- 
ships. As expressed by Tomlinson (2017): 


[t]he comparative dimension of universities’ performance in the form of league tables and 
information sets is seen as crucial in information [of] student “choice”. Consumerism is 
portrayed as part of an increasingly subservient and defensive institutional climate that 
reflects a largely reactive position of professional accountability to external stakeholders’ 
demands for transparent forms of provision that meet instant gratification needs. (Tomlinson 
2017, p. 454) 


Such “needs” are reinforced, shaped, and channelled by media reporting and 
media displays of evaluation information, not least when the evaluation information 
can be ranked and league tabled in a format fitting the “media logic” (Altheide and 
Snow 1979). Media outlets thus actively contribute to constructions of meaning 
(Thomas 2009; cf. Dahler-Larsen 2012 on constitutive effects). In a way, both the 
media and public agencies that rank and evaluate HEI performance can be argued to 
be aligned with a public mission to scrutinise — they are often perceived as defend- 
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ers and enhancers of the public interest. But they also have fundamental differences. 
The national agency is an integral part of the state and its bureaucracy, in the form 
of public administration. News media are not bound by such formal or legislated 
obligations. 


Logics of Appropriateness 


In the language of normative institutionalism, there is a certain logic of appropriate- 
ness in operation: of taken-for-granted routines and operations defining action and 
relations (March and Olsen 1989, 2006), “collections of interrelated rules and rou- 
tines that define appropriate action in terms of relations between roles and situa- 
tions” (Peters 1999). The “media logic” (Altheide and Snow 1979) is signified by 
providing fast and easy access, readership-friendly accounts, etc. The media logic 
tends to favour “unambiguity, episodic frames (...) to focus on conflicts and has a 
prevailing negative bias (...) designating roles of heroes, victims and villains” 
(Thorbjørnsrud 2015, p. 181). 

The media intersects — and perhaps collides — with bureaucratic ideals or visions. 
Such bureaucratic virtues are often conceptualised in terms of, for instance, impar- 
tiality, correctness, neutrality, or adhering to regulations. These key terms describ- 
ing bureaucratic ideals may not be easily aligned with the media logic, at first glance 
(Thorbjørnsrud 2015). In this chapter, some of the instances when media logic and 
bureaucratic public agency work intersect will be pinpointed and examined through 
the evaluations performed within the 2011-2014 EQA. 


The Public Agency-Media Relationship 


At the outset and from the agency perspective, at least three strategies or responses 
can be identified for the agency/HEI-(news) media relationship: to accommodate, to 
be proactive, and/or to be protective (cf. Table 1): 


Table 1 Public administration and news media interactions: strategies and stances 


Adapting and balancing | Bureaucratic branding Information control 
Accommodating: Providing | Proactive: Actively Protective: Controlling and 
information that accommodates the | promoting positive and steering within the 
formats and timing of the media | favourable images of the institution before it goes 
logic as well as short, quick | agency/institution; pitching | public, to secure uniform 
answers, in combination with positive news; spreading the | and coherent messages 
upholding bureaucratic ideals of |image of a well-run, | across levels and subunits, 
correctness and accuracy (to a | goal-oriented, and rational _| etc. and to control and 
greater or lesser extent) system, successful steering, | streamline what reaches 

| etc. journalists 
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These three strategies, along with their compatibility with the media logic of 
appropriateness versus the bureaucratic ideal, will first be discussed in relation to 
the attempted framings and media communications by the agency and the selected 
HEIs (see below). Secondly, they will underpin the analysis of HEI-media interac- 
tions in the context of results from two national subject area evaluations performed 
by the responsible national agency within the framework of the 2011-2014 EQA 
system. 


Cases and Empirical Sources 


The study focuses on a selection of HEIs. These cases have been selected as a part 
of the research project to represent different overall outcomes in the national evalu- 
ations (in terms of share of study programmes judged as “inadequate”, cf. Ericson 
2014) and different institutional characteristics (university versus university col- 
lege; old established institution or younger). This multiple case study (Stake 2006) 
thereby came to include four HEIs that displayed variety in terms of age, size, spe- 
cialisation, and geographical location and were characterised by different contex- 
tual conditions as well as different outcomes to the national quality evaluations (see 
also Table 1 in the chapter “Enacting a National Reform Interval in Times of 
Uncertainty: Evaluation Gluttony Among the Willing”). 

In brief, Orion is a large old university with several faculties and subject areas 
that overall did well in the evaluations on an aggregate level, Hercules is an old and 
specialised university with one faculty and mainly professional programmes that 
did not do as well, Virgo is a comparably recently established university college 
with mainly professional programmes that did well in the national evaluations, and 
Pegasus is a comparably recently established university with both professional and 
academic programmes and courses that overall did not see much success in the 
national evaluations. 

This chapter is based on a range of different empirical sources: first, press 
releases and evaluation reports from the responsible national evaluation agency 
(SHEA) and its predecessor (SNAHE) and second, press releases and information 
from the four HEIs’ webpages, such as communication policies and records/ 
archives of press releases. Some press releases were hard to retrieve, as they date 
back several years, and the websites may not be updated. For some of these 
instances, a report from Academic Rights Watch (2015) — a foundation instigated to 
safeguard academic freedom and rights — has been valuable for tracing and obtain- 
ing press releases from the four studied HEIs. Third, media articles from the 
Swedish media database, Mediearkivet, were analysed, using search terms such as 
the HEI’s name, the evaluation agency’s name, and the subject area. To ensure con- 
fidentiality for the participants in the HEIs, the newspapers are not explicitly men- 
tioned or listed as references. These searches did not provide an overall picture of 
the media debate in general but were focused on the particular HEIs and the selected 
subject areas. This means that the more general and national debate going on at the 
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time (cf. chapter “Hayek and the Red Tape: The Politics of Evaluation and Quality 
Assurance Reform — From Shortcut Governing to Policy Rerouting”) is not included 
as a result of the particular selection criteria employed in this media study. Finally, 
the chapter also uses data collected within the wider research project that were ana- 
lysed to provide additional contextual understanding of the four HEIs, including 
interviews with vice chancellors and senior management at the faculty level at the 
four HEIs. 

The data has been ordered by thematic coding. For the press releases, the themes 
were derived from the main attempted “pitch”, i.e. if the texts were mainly high- 
lighting positive aspects or negative aspects, or if they were mixed press releases 
(with both top and low performers highlighted by the agency) or neutral press 
releases, with no value statements or attempted anglings present. In the primarily 
local media reportings, the thematic coding targeted how the media angling was 
related to the HEI press release, who was allowed to speak (by quotes in the arti- 
cles), and in what role and context (for instance, as shamed or blamed or as heroes/ 
winners) (cf. Ekecrantz and Olsson 1991). 

Before turning to the empirical study, the next section briefly provides some 
additional background and information on the processes and rationale of the national 
agency quality evaluations within the 2011-2014 EQA system. 


The Policy Context and the 2011-2014 EQA System 


In general, Sweden has followed the same kind of overall reform that other EU and 
OECD countries have undergone. The keywords of these reform efforts include, for 
instance, efficiency, transparency, customer orientation, and accountability, with an 
intensified focus on comparisons, data, indicators, and reinforcement of external 
evaluation. Drawing on Karlsson et al. (2014), the following main traits can be 
highlighted in the recent Swedish higher education policy context. The first are 
aspects relating to the HEIs and their relationship with the external environment. 
Reforms have been launched that can be expressed in terms of marketisation and 
competition, including an intensified hunt for resources. Even if attending Swedish 
higher education is free of charge, HEIs receive public funding based on the number 
of students they enrol. In addition, the introduction of tuition fees for non-EU citi- 
zens in 2011 marked a break with the former non-fee-paying system of Swedish 
HE. There is also an intensified emphasis on ranking and league tables, aimed at 
steering and guiding the potential “customer”, in the wake of these general develop- 
ments (see, for instance, the development of a Web-based tool to be used for com- 
parisons between HEIs and with regard to their quality, as assessed in the 2016 EQA 
system) (SHEA 2018). 

Secondly, reforms have also targeted what happens inside the HEIs, by reform- 
ing university management and organisation. Overall, several recent Swedish 
higher-education policies are aimed at HEIs taking a more active, self-governing 
role, which is assumed to lead to increased efficiency and improved outcomes. The 
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so-called autonomy reform (Government Bill 2009/2010:149) is one example of the 
striving to make HEIs more independent and self-governing in some respects, such 
as with regard to internal organisation and hiring and promotion of staff. In these 
reform efforts, several performance-monitoring measures have been instigated by 
the state, such as audits, indicators, and intensified external evaluation. Bibliometric 
follow-up of research and the 2011-2014 EQA system are just two examples. As we 
saw in the previous chapter, the latter has also contributed to sparking a wider debate 
about trust and accountability in Swedish HE (see also Karlsson et al. 2014; cf. 
Kettis and Lindberg-Sand 2013). 

As described in the chapter “Hayek and the Red Tape: The Politics of Evaluation 
and Quality Assurance Reform — From Shortcut Governing to Policy Rerouting”, 
Sweden introduced a highly debated evaluation framework for assessing quality in 
higher education in 2011 that was explicitly focused on results and student out- 
comes: the 2011-2014 EQA system. Implementing this framework led to the 
Swedish agency being excluded from the European Association for Quality 
Assurance in Higher Education (ENQA), of which Sweden originally had been one 
of the founding members (see chapter “Europe in Sweden” and Segerholm and Hult 
2015). Among other things, the Swedish EQA system at the time did not consider 
HEIs’ internal quality-assurance procedures. It only focused on student results and 
outcomes, which the ENQA criticised, and turned out to be an infected and decisive 
issue in the Swedish policy discourse. In addition, the fast-paced “shortcut” policy 
process and detailed instructions set up by the ministry were an additional and 
important part of the criticism, targeting both content and the process by which it 
was designed and approved (cf. chapter “Hayek and the Red Tape: The Politics of 
Evaluation and Quality Assurance Reform — From Shortcut Governing to Policy 
Rerouting”). 

The 2011-2014 EQA system targeted study programmes that could lead to the 
award of a first- or second-cycle qualification and assessed the extent to which the 
students’ learning outcomes corresponded to the intended learning outcome: the 
main assessment point was students’ independent projects (called final degree proj- 
ects). The evaluation resulted in a final overall grade of very high, high, or inade- 
quate quality. The lowest grade meant a follow-up by the agency, with the possibility 
of a revoked entitlement to award degree qualifications (SHEA 2014a). Extra fund- 
ing was given to top performers (highest grade). The evaluations in full, and not 
only the final grade, were then made public on the agency’s website, most often 
followed by a press release from the agency. 

The media searches and the data collection in this chapter target the 2011-2014 
EQA system as well as the period 2014-2016, when SHEA was preparing for a new 
EQA system but continued to carry out follow-ups, which we label a “reform inter- 
val” (see the chapter “Enacting a National Reform Interval in Times of Uncertainty: 
Evaluation Gluttony Among the Willing”). To specify, no additional or new evalua- 
tions of study programmes were undertaken during this period, but the HEIs that 
had failed and received the inadequate quality grade were reassessed to see if they 
could now be passed or not. 
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Evaluation Agency Framing of the Evaluation Results 


As stated in the introduction to this chapter, media display is indeed high stakes for 
HEIs. But media reporting is important to the evaluation agencies as well. This is 
visible in the SHEA’s annual report of its activities, directed to the government and 
the general public: 


Our work receives much external attention and interest. This is obvious, not least, by the 
fact that media reporting on SHEA had a three-fold increase compared to last year. The 
agency’s work is important and contributes to improving Swedish Higher Education. 
(SHEA 2015, p. 4) 


As the above quote illustrates, media coverage is in itself displayed as a sign of 
policy and agency success and is linked to the mission to improve HE. As Gewirtz 
et al. note, “(c)ertain policies require the demonstration of progress and success and 
(...) this in itself becomes an intrinsic feature of the policies” (Gewirtz et al. 2004, 
p. 327). Given the importance of the media and display, and also as a means of actu- 
ally making the policy “work”, the next section focuses on the evaluation agencies’ 
media communication by analysing their press releases. 

Relevant press releases issued by the responsible agencies (the SNAHE and then 
the SHEA) were collected for the years 2011-2016 (N = 36) on the topic of evalua- 
tions of study programmes. As previously mentioned, the analysis sought to identify 
their framing by categorising them as mainly positive, negative, mixed, or neutral 
(Table 2). 

In the positive category, many releases were about follow-ups conducted one year 
after a programme had been judged as inadequate. Very few programmes were ever 
revoked of their rights to issue degrees. The positive evaluation results — since 
almost all of the programmes passed the follow-ups — also function as an implicit 
justification of the agency (compare to bureaucratic branding), as “now” there has 


Table 2 SNAHE and SHEA evaluation results press releases, 2011-2016 (N = 36) 


Positive | A majority of the follow-ups are in this category; almost all reassessed programmes l 
(n=6) were judged as high quality, thereby shedding positive light on the agency as well as 
claims that the 2011-2014 EQA system had in fact resulted in a “significant raise in 
quality” 


Mixed Expressed as, for instance, “mixed quality in artistic training” (and several other 
(n=11) |subjects were expressed in the same way), i.e. the releases mention both negative 

and positive results. Often, these press releases begin with and emphasise the failed 
and lacking programmes and HEIs and then mention top performers 


Negative | Mainly focusing on and framing the release in terms of failures, such as “several 
(n=6) programmes failed”, “many programmes failed”, or “midwife training programmes 
are insufficient”. (Note: none of these midwife programmes earned the highest 
grade, as 15 were judged as high quality, and only 3 as lacking quality, but the 
headline of the press releases still concluded “insufficient”’) 


Neutral These press releases are stating a new evaluation that has been completed and 
(n= 13) | published, no quotes that are assessing/valuing the results, with headlines such as 
“New evaluations”, “Evaluations now finished’, etc. 
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been “a significant increase in quality”, as the headline may say, i.e. the agency and 
its evaluation work matter, and they raise quality. The analysis of agency press 
releases thereby illustrates how a favourable bureaucratic branding strategy is dis- 
cernible within this format of directed media communications. This can be inter- 
preted within the performative dimension of agency reputation management 
(Christensen and Gornitzka 2018), in which the agency is delivering outcomes 
relating to its core mission. 

The analysis also identified mixed press releases, which highlighted both nega- 
tive and positive aspects. These releases often begin by describing what is lacking 
and how many and what HEIs failed or were found inadequate, but the releases also 
mention top performers and those who got the very high quality grade, which 
received an extra government grant — as a way to single out the “heroes”. 

Looking at the negative press releases, they are in themselves attractive to the 
media logic; “failures”, etc. give leeway to feelings of sensation and even scandal, 
victimised students, and evil, underperforming HEIs. The midwife training press 
release can serve as an example: “Midwife training programmes are inadequate” 
headlined the press release, but in fact only 3 programmes were judged as inade- 
quate, and 15 passed with high quality. The neutral press releases were the most 
common framing. These releases merely state that “a new evaluation is finished” 
and list the results, without any quotes, opinions, or statements from agency 
representatives. 


Media Coverage of Two Subject Areas 


In the coming sections, some results on two subject areas are presented in more 
detail, in the particular context of the four studied HEIs. The first area, education, 
had a mixed press release, and the second, specialist nursing, had a negative one. 
Both categories can be said to connect well with the media, based on what the media 
logic would amplify and value (negative angles, villains, deviations rather than suc- 
cess stories, etc.). Furthermore, these subject areas concern professional degrees 
and are large programmes that are important for meeting the demand for qualified 
staff in the whole of Sweden, making them courses and programmes of particular 
public and political interest. 


Education 


The education subject area comprised programmes in education, specified as also 
including didactics, educational leadership, and psychotherapy. The SHEA launched 
a “mixed” press release: 


One third of all programs, 19 out of 57, got the grade inadequate quality. These programs 
are offered in 12 out of the total 24 HEIs assessed in this evaluation. The shortcomings are 
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often connected to lacking scientific quality in students’ independent degree projects” (...) 
Five programs are judged as very high quality and will receive an extra quality grant from 
the government, and 33 programs are of high quality. (SHEA 2014b) 


The table below summarises the HEIs’ results in the evaluation, how they pre- 
sented the evaluation results themselves, and the (mainly regional) media coverage 
(Table 3). 

In Orion’s case, the “inadequate” grade was not mentioned in the HEI press 
release, but it was picked up by the student union journal. Other than that, there was 
little media attention. Hercules chose to communicate and highlight its very high 
grade (not the high grade), but there was not much media response and attention. 
Virgo had one evaluated programme that got the grade “high quality”, which was 
also highlighted in a press release. This was followed by a “kind” and smooth fol- 
low-up and exposure in a local newspaper that even “advertised” how students 
would go about applying to this successful programme (which did not even receive 
the highest grade), by stating the deadlines for applications, who to contact, etc. 
Pegasus had mixed results, and the regional paper chose to highlight the good grade 


Table 3 Subject area: Education 


HEI and 

evaluation HEI news and/or press 

results release Media coverage 

Orion “Good grades for The student union journal reported that “Education 


Six evaluations: 
Very high: 4 
High: 2 
Inadequate: 1 


[Orion’s] certification 
degrees in didactics, 
educational leadership, 
and education” 


has inadequate quality”, relying on the SHEA press 
release and an interview with an Orion 
representative saying, “we are taking this very 
seriously”, and that Orion has already started 
improvement work. 


Hercules “The certification No reporting with Hercules as a focal point (this 
Three degree in HEI is only mentioned in articles that report a full 
evaluations: psychotherapy gets the | list of all evaluated HEIs). 

Very high: 1 grade of very high 

High: 2 quality 
Virgo “[Virgo’s] certification | A regional paper ran the headline “High quality at 


One evaluation: 
High: 1 


degree in education is 
of high quality” 


the university” and quotes the vice chancellor from 
the HEI press release, along with another positive 
voice from a Virgo representative. Finally, the article 
provides an “advertisement” to prospective students 
by stating, “the next opportunity to apply for the 
programme is...” 


Pegasus 


Two evaluations: 


High: 1 
Inadequate: | 


“Mixed results in 
evaluation of 
education” 


A regional paper stated that the “second cycle 
degree gets good grade” and “it was both good and 
bad grades when courses in education were graded 
and evaluated”. A Pegasus representative is quoted 
saying they have already started the improvement 
work and will continue it and take the criticism 
seriously. The final sentence reiterates the grade 
“high quality” for one of the evaluated degrees. 
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in its headline. It also allowed Pegasus representatives to elaborate and assure the 
readers that it was working to improve. 


Specialist Nursing 


The results from the evaluation of specialist nursing and care led the SHEA to 
launch quite a negative press release: 


The SHEA has finalised a comprehensive evaluation of programs in specialist nursing and 
care. Seventy-nine out of 134 programs are judged as having inadequate quality, and their 
right to reward degree qualifications is now questioned. This is a serious situation, and the 
HEIs of course need to rectify this (...), says University Chancellor Harriet Wallberg. 
(SHEA 2014c) 


This particular evaluation made it to the national news and was distributed by the 
main national news agency TT. It then travelled to other regional media outlets. 
Table 4 below shows how the HEIs communicated the evaluation results, alongside 
whether and how the evaluations were picked up and framed by (the regional) news 
media. 

As displayed in the table, Orion attempted a framing, but the regional paper went 
for the national news agency’s formulations. Hercules had very bad results and did 
some damage control attempts in its releases, by being open and receptive and stat- 
ing it is working hard to improve. This made it to the national media. Virgo did well 
and took advantage of it. The “hero story” it launched successfully came through in 
the subsequent media coverage. Pegasus had three regional outlets that covered the 
story. Two did their own articles, while one went for the national TT news agency 
text. The two articles written by local journalists let Pegasus representatives explain 
and expand that they were working hard to improve. 


Media Communication, Agencies, and Evaluation 


The analysis in this chapter showed that the agencies’ press releases varied in their 
“pitch” and that they are displaying signs of bureaucratic branding, such as in the 
follow-up press releases from the national evaluation agencies (compare to perfor- 
mative reputation management, Christensen and Gornitzka 2018). The negative 
releases also showed some tendencies to scandalise, as illustrated by the midwife 
evaluation press release. Press releases from the HEIs also varied in their attempted 
angling, unsurprisingly attempting to frame the issue in a favourable and construc- 
tive light but not always. There was noticeable variation in this regard. 

Throughout the study, the evaluation agencies (the SNAHE and the SHEA) 
appeared to be conceived as an untouchable form for check-up. They are perceived 
as credible, reliant, and good sources of non-biased information that the media is 
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Table 4 Subject area: Specialist nursing 
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HEI and 
evaluation results 


HEI news and/or press 
release 


Media coverage 


Orion 

Nine evaluations: 
High: 3 
Inadequate: 6 


Hercules 

Ten evaluations: 
High: 1 
Inadequate: 9 


“Many specialist nursing 
programmes are 
questioned”; 

“A few was judged as 
lacking quality”; “We are 
happy to see that several 
programmes were graded 
as high quality on some 
of the assessment points” 
(faculty-level chair) 


“Special nursing 
education is questioned in 
a new evaluation”; 
“Scientific quality is 
lacking”; 

Quoting a dean: “We 
should not offer 
education that is lacking 
quality. We will work to 
ensure that all our 
programmes are of high 
quality” 


A regional newspaper stated, “Major flaws in 
nursing education”. 

This text is from the national news agency TT, 
quoting the chancellor emphasising the 
seriousness of the situation and that this was a 
national problem that should be seen as an issue 
for the government and not just the HEIs. The 
final part lists Orion’s results. 


National TV news: “Special nursing education 
heavily criticised”. Hercules’ representative was 
interviewed and said it took this seriously and 
needed to make an extensive improvement 
effort, portraying Hercules as already on the 
route to successful improvement. 

National newspaper: “Alarm: Risk for several 
programmes to be cancelled”, “Can be 
catastrophic”, and “Probably the most severe 
criticism ever raised in Swedish higher 
education evaluation”. The chancellor is quoted, 
and the article was distributed via TT. 


Virgo “District nursing Regional newspaper: “The university gets the 
Two evaluations: | programme at the highest grade”. The vice chancellor is quoted 
Very high: 1 absolute national top elite | discussing the good quality and many students 
High: 1 level” applying for the programme 
Local radio: Education at Virgo is highlighted as 
a good example. 
Pegasus “Questioned quality in Three regional newspapers: (1) “Nursing crisis 
Seven specialist nursing can get worse”, with quotes from a Pegasus 
evaluations: programmes” representative saying it will work hard to rectify 
High: 1 the problems and is not worried — think it will 


Inadequate: 6 


pass the follow-up. (2) “Inadequate education”, 
with quotes from a Pegasus representative 
assuring it is taking the results seriously. (3) 
“Major flaws in Pegasus nursing programmes” 
from TT but with angling in headline and about 
the end to the regional area and to Pegasus (like 
the Orion article). 


depending on. This finding is similar to a study on another Swedish agency, the 
Swedish Schools Inspectorate (Rönnberg et al. 2013). The study reported in this 
chapter highlighted how these central and perceived credible state actors chose to 
frame their press releases and communicate with the media. These attempted fram- 
ings easily flow through and are not generally subjected to additional media scrutiny. 
The mission of independent media scrutiny was not highly pronounced in the anal- 
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ysed material, and the media seldom reprocessed evaluation results data or high- 
lighted alternative ways of interpreting them. This gives leeway to both successful 
pitching to the press and for opening up for bureaucratic branding activities. 


Bureaucratic Logic of Appropriateness in Agency Branding 


In balancing and adapting the bureaucratic logic to the format of the media logic, 
the agencies and, to some extent, the HEIs were seemingly successful in how they 
met and interacted with the (regional) media. The findings point to some instances 
where favourable bureaucratic branding of the HEI was amplified by supportive 
local media reporting, without the media asking for additional sources and/or infor- 
mation, such as in the Virgo hero story and, even if less pronounced, in the Pegasus 
case. But the pattern is not clear-cut and may depend on the HEIs’ different roles 
and academic profiles. Virgo and Pegasus are both regional universities with impor- 
tant ties to the local and regional area. 

The findings also point to an adapted form of packaging and to selling “bureau- 
cratic” information, which make the mediation and translation from the agency/HEI 
to the media particularly smooth. The bureaucratic values and ideals function as an 
important “currency” in the packaging and selling of information via media. In fact, 
the bureaucratic logic of appropriateness is a part of the bureaucratic branding. It is 
a good media pitch, as it is perceived as credible and linked to perceptions enhanc- 
ing its legitimacy. This is all managed via well-skilled communication offices in 
both the agencies and the HEIs. Based on these findings, we may be witnessing a 
“professionalisation” of public agency-media relations, including bureaucratic 
branding activities (cf. Christensen and Gornitzka 2018). Like the spread of mana- 
gerialism, this study points to the important role played by a professionalised cadre 
of “bureau-branding” employees at universities and public agencies that are work- 
ing to design, manage, and steer external and internal communications. These 
employees are professionally “pitching the press” under the legitimising shield of 
the bureaucratic logic. 


What Happened to Critical Debate? 


An important silence concerns the absence of critical debate on the evaluation 
framework’s validity. Very little of the criticism and debate about the 2011-2014 
EQA system (see the chapter “Hayek and the Red Tape: The Politics of Evaluation 
and Quality Assurance Reform — From Shortcut Governing to Policy Rerouting”) 
was visible in this study. However, the fact that the empirical material does not 
cover the overall media discourse in general but rather has pinpointed certain HEIs 
and two evaluations in particular needs to be reiterated. Even so, this material dem- 
onstrated very few traces of the intense critical debate on the 2011-2014 EQA 
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system. This debate seems to have been effectively silenced in the reporting on 
evaluation results when the system actually was in operation. The framework that 
produces these results is questioned in what appears to be “a parallel debate” — not 
intersecting with the reporting on these evaluations’ actual results. 

Journalists do not use the angle of the political debate and the criticised evalua- 
tion system design in these reports. The interviewed HEI representatives do not 
bring it up either, according to how they are cited in the analysed media materials. 
The HEIs and their representatives are not attempting to pitch the criticism of the 
system as a possible defence and/or counter argument when responding to individ- 
ual programme evaluations’ results. Largely, the approach taken when an HEI is 
judged inadequate is for HEI representatives to be submissive and to show compli- 
ance and willingness to adapt to the SHEA and improve but assume that they will 
rectify the situation and meet the standards, assuring people that work is already on 
its way — a calming message to the public. When objecting, HEI representatives 
may risk being judged as having something to hide, they may come across as unre- 
ceptive, and any critique can be turned against them and make the HEI look defen- 
sive, stiff, and unwilling to improve. In this way, the performative agenda is 
narrowing the space of what can be said. 


Finally 


This chapter empirically illuminated some of the processes in which high-stakes 
national evaluations, PR strategies from the HEIs, and the media logic meet and 
intertwine. It gave some empirical accounts on how processes of mediatisation are 
taking place in and through agencies and highlighted some important interdepen- 
dencies that need to be critically discussed in relation to their possible constitutive 
effects (cf. Dahler-Larsen 2012). Not only is the existence of national EQA systems 
forming certain perceptions of what good higher education is “supposed to be” in 
this case, but the mediatisation of these evaluation results and central stakeholders’ 
navigation of these processes are also important parts of these perception forma- 
tions and the potential constitutive effects of quality assurance and other forms of 
evaluations. The way the media uncritically presents evaluation results from the 
EQAs may also contribute to the image of them as unproblematic and objective 
ways of measuring and controlling for “quality” in higher education. 

Having outlined some of the mediated representations of actual evaluations con- 
ducted within the 2011-2014 EQA system, the next chapter discusses the period 
when the national agency and the HEIs prepared for the new, yet not formally 
decided upon, 2016 EQA system that was to replace the debated 2011-2014 EQA 
system. We will describe and analyse how the four case HEIs — Orion, Hercules, 
Virgo, and Pegasus — navigated, developed, and designed their own internal quality 
assurance systems. 
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Enacting a National Reform Interval A 
in Times of Uncertainty: Evaluation gest 
Gluttony Among the Willing 


Joakim Lindgren, Linda Rönnberg, Agneta Hult, and Christina Segerholm 


Abstract This chapter offers an empirical illustration of the governing-evaluation- 
knowledge nexus by pinpointing a particular situation, a pause between two national 
evaluation and quality assurance (EQA) reforms, while a new national system was 
being planned and prepared, but its final design was not yet decided upon. This 
situation — unusual in the Swedish higher education policy context — adds uncertainty 
to the situation and opens a potential space for policymakers and higher education 
institutions (HEIs) to navigate. We draw on interviews and documents collected 
from four HEIs during this reform interval. We analyse and discuss how the four 
institutions navigate, coordinate, mobilise, copy, and learn in a situation without a 
formal national EQA system in place but in which the wider higher education policy 
context is deeply infused with contemporary trends and international policies and 
ideas on quality assurance (QA). We found that context and institutional 
preconditions set their mark on the work undertaken during this interval. We also 
discerned tendencies of homogenisation and isomorphism. Finally, we highlight the 
tendency of further expansion of EQA activities. 


Introduction 


As we presented in the chapter “National Evaluation Systems”, national require- 
ments for HEIs to install and maintain internal quality assurance (IQA) work have 
been in place for around 25 years in Sweden. A number of national EQA systems 
have existed for evaluating quality in higher education during this period (cf. 
Segerholm et al. 2014). Previous chapters have shown how evaluative activities in 
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higher education have expanded into internal institutionally based and national EQA 
systems as an evolving evaluation machinery and that such means of governing are 
now an important part of policymaking and educational practice (Dahler-Larsen 
2012; Westerheijden et al. 2007). We have also drawn attention to the European 
dimension of EQA. In the chapter “Europe in Sweden”, we described how the 
European Association for Quality Assurance in Higher Education (ENQA), in coop- 
eration with other European organisations for higher education, has promoted the 
development of national and local systems for higher education in Europe since the 
beginning of the millennium (Standards and Guidelines for Quality Assurance in 
European Higher Education Area (ESG) 2015). We also analysed the extensive 
national debate from 2014, when the Swedish agency was no longer accepted as a 
full member of ENQA because the 2011-2014 national EQA system was not con- 
sidered as meeting the ENQA standards. Simultaneously, heavy criticism from aca- 
demics and students in this strictly outcome-oriented national system led the 
government to terminate it in 2014 and appoint an investigator to look into these 
issues (Segerholm and Hult 2015). These events came together in a quite unusual 
situation as a break from the way reforms are normally prepared and implemented in 
Sweden — i.e. well in advance of implementation and following traditions of rational 
planning and social engineering, in which EQA systems have replaced one another 
sequentially with no evident “gaps” between systems (see the chapter “Governing 
By Evaluation: Setting The Scene”, the section on the Swedish case). This situation, 
which we have characterised as a “national reform interval’, signifies the time 
between the termination of the 2011-2014 national EQA system and the formal 
parliamentary decision in March 2016 to approve the 2016 EQA system. It also 
includes the work of the responsible agency, the Swedish Higher Education Authority 
(SHEA), until the final and elaborated upon system was presented in June 2016, 
which marked the end of the reform interval. Earlier, in March 2015, the government 
sent out a memorandum for referral to HEIs, students, and university teacher unions 
(Ministry of Education 2015). This government document gave the HEIs certain 
cues about what EQA reform to expect, should the parliament approve it. 

This chapter offers an empirical illustration of governing by evaluation in higher 
education in this particular situation. This reform interval implied a pause or break — 
a situation of some uncertainty but also some expectations concerning the content 
of the forthcoming reform. This particular situation, we argue, opened a potential 
space for the HEIs: How would they navigate the situation without an explicit 
reform decision to relate to while receiving certain government cues about what 
would be waiting in the forthcoming reform? Would HEIs wait until the final 
parliamentary decision was made, or would they start to prepare and develop 
routines for QA in certain directions while remaining unsure whether they had to 
change it or not? The aim of this chapter is to illuminate the work during this reform 
interval at the national/state levels and among the HEIs.' The following questions 
guide the chapter. 


'The chapter draws on our joint empirical research reported and presented in the following confer- 
ence papers: Lindgren and Rönnberg (2017), Segerholm et al. (2016), and Segerholm and Hult 
(2015). 
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e What processes and work were undertaken in the national policymaking arena 
during the reform interval? 

e What policy processes and practices (if any) do the HEIs have for handling and 
responding to the reform interval situation? 

e How can such processes be understood in terms of education governing? 


First, we outline our conceptualisation of the reform interval and how it can be 
theoretically understood as work, processes, and practices within the levels and 
domains of the state and HEIs. We will then address the first question to describe the 
reform interval and take a more elaborate look into the frame that the government 
gave in the above-mentioned 2015 memorandum. This section illustrates what the 
HEIs had to relate to during the reform interval. A section on the design, methods, 
and materials that we used in our empirical study of four HEIs will follow this 
illustration. We will then analyse the four cases, with a particular focus on their 
work with QA during the reform interval, before concluding with a discussion of 
our main findings. 


Mind the Gap: Conceptualising the National “Reform 
Interval” 


The notion of an interval indicates a time gap or an intervening time, a pause, or 
break in activity. To what extent is the present empirical case that we have labelled 
a “national reform interval” tangent to such notions? What evaluative activities of 
the state were paused (and what activities were not)? What aspects of work with the 
design, development, and enactment of IQA systems could be carried out in HEIs 
without detailed information about the standards against which it would be assessed? 
Did the HEIs respond differently to the lack detailed information? Did a space, 
room for manoeuvring that entails a condition for freedom and autonomy, open 
between two objects (policies)? Perhaps such an interval is not something out of the 
ordinary but instead is a rather common situation in public administration. If a new 
policy proposal is put “on hold” until it is enacted legally, the educational sector 
might expect that the temporary proposal will likely later be enacted as legislation, 
and, in that case, the situation would not be that extraordinary. In addition, European 
standards in the form of the ESG were firmly in place, offering guidance for Swedish 
HEIs, during the interval (Grifoll et al. 2012). Here, notions of “embeddedness” 
(Jacobsson et al. 2015) and the intertwined relationships between state agencies and 
international organisations are crucial. 

We suggest that a reform interval is a general conditional situation of uncer- 
tainty — a rather common, if perhaps intensified, situation. Moreover — as Jacobsson 
et al. (2015) noted — the values of professionalism and mutual trust typically char- 
acterise Swedish state governance. Governing in the form of “micro-steering” can 
thus rely on subtle and informal means without giving the bureaucracy extensive 
and detailed guidance. In this case, an interval may imply “intensification” and reac- 
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tions to uncertainty that raise empirical questions. Here, anticipation and cue giving 
have been used to explain how agency staff predict or second-guess political inten- 
tions and act on these hunches (Jacobsson et al. 2015). 

If activities — such as the design, development, and enactment of IQA systems — 
depend on detailed prescriptions, HEIs could choose to await such information. If 
the SHEA will externally evaluate them in a certain way, the HEIs may well await 
such evaluation criteria and then design their IQA systems to target such demands. 
The interval is an absence of a national policy, with much uncertainty about future 
policies along with a strategic incentive to “wait”. Anticipatory behaviour is 
associated with risk (Jacobsson et al. 2015), and this in turn raises important 
analytical questions, such as those regarding how evaluation-making devices and 
arrangements function in the absence of stated and formal national policies. In other 
words, does friction, delays, or “loose-coupling” exist between policy and 
educational management? 

The reform interval provided us with opportunities to empirically study issues of 
governing in the making. Governing, as we noted earlier in this book, includes 
processes in which several instances and actors are involved in forms of work — 
including interpreting, negotiating, translating, and enacting policy (Clarke 2015). 
The stress on the actual work of governing and “doing” policy is arguably also 
consistent with the concept “enactment” (cf. Ball et al. 2012) in processes of policy 
interpretation and translation. Translation, as Latour (2005) pointed out, involves 
processes in which all actors and artefacts mutually transform and are transformed 
by the environment that they engage in. In this context, we are particularly interested 
in questions about whether — and, if so, how — enactment during the interval produces 
“irreversible interactions”, i.e. enrols durable changes in social and material 
elements related to core activities within HEIs. 

Our starting point is that translations of QA appear in “different but similar set- 
tings, where local resources, material and human, and diffuse sets of discourses and 
values are deployed in complex and hybrid processes of enactment” (Ball et al. 
2012, p. 6). This means that the local context is of importance in understanding 
policy processes and governing during the interval. The overall characteristics of the 
Swedish political-administrative system, which manifested in the recent EQA 
reform processes that we analysed in the chapter “Hayek and the Red Tape: The 
Politics of Evaluation and Quality Assurance Reform — From Shortcut Governing to 
Policy Rerouting”, include evident attempts to produce and establish policy through 
dialogue. This makes us devote particular attention to the inherent complexities of 
governing, given that various actors formulate policy at various levels. Complexity, 
however, does not contravene with the fact that many Swedish HEIs — at least on the 
surface level — appear quite similar: They appear as well-integrated organisations 
with modern, systematic, and thorough practices in terms of routines, strategies, and 
documentation that aim to enhance the quality of research, education, and 
cooperation with the surrounding society. 

Acknowledging such similarities, we seek to explore how forms of internal HEI 
work on QA are enacted and evolve through processes of cooperation, imitation, 
editing, or as fashion (Sahlin and Wedlin 2008). Isomorphism (DiMaggio and 
Powell 1983) offers a classic attempt to analyse aspects of such processes in terms 
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of three mechanisms: for example, coercive isomorphism, which stems from 
external political pressure from national and international policies and organisations 
in this case. On the other hand, mimetic isomorphism emphasises organisational 
responses to uncertainty. This leads to the assertion that if the HEIs experience the 
studied reform interval as uncertain, they might respond with mimicry. The final 
mechanism, normative isomorphism, relates to professionalisation. In this context, 
we are interested in the work, knowledge, and networks of key actors at the HEIs 
and the extent to which such factors produce relative homogeneity within the sector 
in the enactment of a national reform interval concerning EQA. Isomorphism in 
turn draws our attention to questions of rationality. As DiMaggio and Powell (1983) 
noted, the adoption of new (fashionable) ideas might provide legitimacy rather than 
improving actual performance, and not all ideas will be rational for all kinds of 
organisations. 

We draw on these theoretical resources to explore policy processes and practices 
during the reform interval. During the interval, little was known about whether the 
new EQA system would lead to the further expansion of evaluative activities or 
whether an eventual expansion would produce better-functioning HEIs. This chapter 
is a contribution to a discussion on these challenging questions. Next, we turn to the 
processes at the national level, including the work of government; an appointed 
single-member commission’s efforts in outlining a new EQA system; and the 
SHEA’s task in finalising the system’s design. 


The Reform Interval: The Government’s Work Towards a 
Parliament Decision 


As described in previous chapters, the centre-right coalition government — in what 
we have labelled a fast-paced and “shortcut” style — decided upon and implemented 
the 2011-2014 EQA system that preceded the reform interval (cf. the chapter 
“Hayek and the Red Tape: The Politics of Evaluation and Quality Assurance 
Reform — From Shortcut Governing to Policy Rerouting”). This was done despite 
early and heavy criticism of its outcome-based orientation and lack of methodological 
rigour. This was, as we also noted earlier, a system that the ENQA criticised because 
it did not live up to the Standards and Guidelines for Quality Assurance in the 
European Higher Education Area (ESG) (2015) and ENQA (2009). The ENQA 
review concluded that the Swedish system failed since it did not evaluate HEIs’ IQA 
system and did not give recommendations for improvement. Also, the national 
agency responsible for the QA could not be considered independent “due to the 
extent to which their procedures and methods, as well as overall aims and objectives 
have been dictated by Government” (SNAHE 2012, p. 23). 

In 2013, the centre-right government decided to terminate the 2011-2014 EQA 
system. In these discussions, the ENQA membership was central in the arguments 
for terminating that EQA system and developing a new one (Segerholm and Hult 
2015). The government commissioned the SHEA to develop such a revised system 
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in 2013, but, in mid-2014, the agency received new directions from the government. 
At that point, the government commissioned an individual investigator, Professor 
Harriet Wallberg Henriksson (later director general of the SHEA), to propose a 
framework for a new national EQA system that would be founded on the following 
basic principles that the government set forth. 


e The system should be cohesive and include the HEIs’ internal QA and SHEA’s 
external audits. 

e The system should control students’ learning (outcomes). 

e The system should improve quality and provide HEIs with incentives and 
guidance. 

e The roles of working life (relevance and employability) and working-life repre- 
sentatives should be more thoroughly included in audits. 

e The role of students should be clearer. 

e European principles on QA should be considered. 

e The system should include sanctions, with possibility of revoking licenses for 
issuing student degrees. 

e The system should consider the autonomy of HEIs and be transparent and explicit 
(Ministry of Education 2015, p. 4). 


The government also directed the investigator to confer with the HEIs, the SHEA, 
students, and working-life representatives. The investigator’s report was eventually 
delivered to the Ministry of Education in December 2014, and, based on that, the 
new Social Democratic-Green Party coalition government (in office after the 2014 
September elections) crafted a memorandum. It was published in March 2015, and 
it set the overall frame for a new national EQA system (Ministry of Education 
2015). In the memorandum, the importance of paying attention to the ESG — along 
with the need to align the new EQA system with the Higher Education Act and with 
the requirements for student degrees, as specified in the Higher Education 
Ordinance — was emphasised on several occasions. The new EQA system was to 
include the following: 


e Accreditation for rewarding degrees and certificates 

e Evaluation of the IQA systems at the universities 

e Evaluation of a sample of bachelor-, master-, and PhD-level study programmes, 
foremost aiming to control the adherence to the Higher Education Act and the 
Higher Education Ordinance and to contribute to improving quality 

e Thematic evaluations — e.g. widened recruitment of students, internationalisa- 
tion, and gender equality (Ministry of Education 2015, p. 3) 


The memorandum proposed that quality should be assessed in terms of student 
attainment and performance, but a description of the preconditions and processes 
that influenced would complement the result. Furthermore, the SHEA would be 
responsible for developing and implementing the new EQA system, based on the 
government’s judgement, laws, ordinances, and principles for QA, which is in line 
with the Bologna Process. The starting point for the memorandum was that all 
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higher education programmes should be evaluated through the universities’ IQA 
systems and the SHEA’s evaluations of them. In the memorandum, it was also 
specified that SHEA, as in previous national EQA systems, should appoint an expert 
panel that should assess the self-evaluations that the HEIs handed in and prepared. 
The SHEA was supposed to later elaborate upon and specify the information and 
aspects of quality that should be included. Interviews were also required to be 
carried out with representatives for the HEIs, including teachers and students. The 
expert panel was supposed to document its assessment in a report that would form 
the basis for the SHEA’s judgement and decision. The memorandum also stated that 
this is an internationally established method for QA in higher education. It also said 
that follow-ups after the assessment can promote quality — as, for example, HEIs 
learn from each other and from observations from the expert panels. 

In carrying out the government mandate to further develop the design for the new 
EQA system’s details, the SHEA organised several hearings with stakeholder 
groups during 2015. These groups included teacher and student unions, vice 
chancellor organisations, and organisations for the private market and public sector. 
However, the parliament was to make the final decision about the system, making 
the HEIs and SHEA insecure about what to actually expect in the end. In December 
2015, the Ministry of Education had processed all incoming referrals on the 
memorandum, and they made very few revisions before sending their proposal to 
the Parliamentary Standing Committee on Education, which finally advised the 
parliament to endorse it. The decision was made in March 2016, with some 
amendments (Government Petition 2015/2016:76; Parliamentary Decision 
2015/2016:155). After this, the SHEA continued to work on the details and the 
additional requirements on the system that the parliament had decided upon. During 
spring 2016, the SHEA had regular meetings and dialogues about its work to design 
the forthcoming EQA system with HEIs, national and international reference 
groups, and groups representing stakeholders. The agency also organised conferences 
among other things to present information and get acceptance for their proposed 
ideas. In June 2016, the SHEA presented their final and detailed design of the EQA 
system and reported this work to the Ministry of Education in early October 2016. 

As the above account demonstrates, the reform interval was not by any means an 
interruption in policy work at the national level. On the contrary, the politicians, the 
public commission, and the SHEA carried out extensive work in designing and 
developing the new EQA system, and this work also fed into the work of other state 
agencies and nongovernmental organisations. Actors and reference groups from 
within academia, including all HEIs and student union organisations; labour market 
actors, including teacher unions; employee organisations; and interest groups, were 
mobilised in rounds of meetings and hearings. 

In the next section, we direct our attention to the processes and practices enacted 
within the HEIs during the interval. Before we enter this exploration, we will say a 
few words about the empirical case study reported in this chapter and briefly touch 
on its design, methods, and materials. 
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Four Cases: Methods and Materials 


The study was designed as a multiple case study (Stake 2006). Four HEIs with vary- 
ing ages, sizes, specialisations, and geographical locations were selected. They also 
represented different aggregated outcomes from the 2011 to 2014 system evalua- 
tions of study programmes. (See Table 1) 

This design arguably fits well with the intended approach for understanding pol- 
icy enactment at HEI level (Ball et al. 2012), since it gives a voice to HEIs that are 
located in quite different contexts and have different institutional set-ups. Our 
selected cases include one old university with a comprehensive number of faculties 
and academic and professional programmes and courses; one new small university 
college with one faculty and mainly professional programmes; one well-established 
specialised university with one faculty and mainly professional programmes; and 
one medium-sized new university with two faculties and professional and academic 
programmes and courses. The four HEIs are geographically dispersed, and, in the 
latest national quality evaluation that the SHEA carried out, they received different 
results — ranging from not good to very good (only one university in Sweden was 
assessed as extremely good) (Ericson 2014). 

We did a total of 16 individual interviews, mainly during spring 2015 and spring 
2016. We interviewed the vice chancellors; central officers who were responsible 
for IQA at the HEIs, including deans and/or other officers at the faculty level; and a 
limited sample of those at the department level who were responsible for IQA. We 
asked the informants about whether and how the HEI prepared for the expected 
national reform, how they perceived difficulties with preparation work, and how 
they envisioned and designed a good IQA policy and practice, and we asked more 
generally about quality and accountability in the HEI. We also collected and 
analysed policy documents of QA work at different levels in the HEIs, the four vice 
chancellors’ blogs, website information, and other types of written materials from 
the HEIs. 


Table 1 Summary of cases in terms of size, age, profile, and education quality 


| Relative size | Relative age Outcome 2011-2014 EQA 
(1—4 from large |(1—4 from old |Profile/ evaluation? (1—4 from high 
HEI to small) to new) specialisation | to low grades) 
Pegasus 2 4 Broad/regional | 4 
University | | 
Virgo 4 |3 Special/ 2 
University regional 
college | | 
Orion 1 1 Broad/ 1 
University | traditional 
Hercules 3 [2 Special/elite l3 
University 


‘Internal ranking from 1 to 4 based on results from quality evaluations by the SNAHE and the 
SHEA between 2011 and 2014 as presented by Ericson (2014) 
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We heuristically used the idea of enactment in organising and analysing the 
material. We concentrated on the questions about what, how, and who and context 
questions concerning the work with the HEIs’ policy and practices during the 
reform interval. We posed the following questions about the collected material. 
What kinds of activities have taken place at the HEIs? What actors have been central 
as driving forces? What kinds of processes were undertaken to justify and legitimise 
the internal work with QA? What cooperation and/or exchange of experiences of 
IQA with other HEIs took place, if any? Did any resistance take place? What parts 
of the organisation are mobilised in IQA work? 


The Reform Interval: IQA Work in Four Higher Education 
Institutions 


In the following, we begin by describing some of the main characteristics of the 
cases (with some restrictions to respect confidentiality) before we move on to 
discuss their activities concerning IQA during the reform interval. 


Pegasus 


Pegasus is a medium-sized and young regional university. Despite being a compara- 
bly small university by Swedish standards, it nevertheless has a quite broad panoply 
of study programmes and courses — including humanities, information technology, 
social sciences, behavioural sciences, media, teacher education, natural sciences, 
health education, science, and technology. Distance education and interdisciplinary 
education are two cornerstones at Pegasus, and it seeks ways to cooperate with the 
local community to promote regional development. Pegasus is organised as a line 
management system with a governing board, vice chancellor, pro-vice chancellor, 
and two faculty boards. In the 2011-2014 EQA system targeting study programmes, 
however, Pegasus was one of the most heavily criticised universities. Almost 50% of 
the programmes or courses were judged as having “insufficient quality”, whereas 
few programmes received the grade “very high quality”. 

Overall, Pegasus took a quite proactive strategy during the reform interval. 
Pegasus and two other HEIs decided to make a joint effort to construct a peer review 
system of assessments for collegial education quality. This endeavour was intended 
to build on ongoing cooperation that had involved aspects of education and 
management, though not yet in the area of education evaluation and assessments. 
Pegasus formed a task force comprising the vice chancellors and additional high- 
level management in early 2015 to investigate whether — and, if so, how — a 
collaborative system of assessing education quality could be designed and 
implemented. The three collaborating HEIs wanted to organise quality assessments 
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of study programmes and courses based on the overall assumption that collaborating, 
developing, and exchanging knowledge and experience between the three HEIs 
would have potential benefits. 

These joint education assessments (henceforth “JEAs”) should however not be 
seen in isolation. The JEAs are considered one part of a wider and more encompassing 
IQA system at Pegasus, which means that the overall organisation of QA includes 
additional dimensions apart from the JEAs. At Pegasus, JEAs are considered 
activities under the heading “quality evaluation”. The other three main headings for 
the IQA system are “pre-emptive quality work”, “continuous quality work”, and 
“quality follow-up” — aiming to cover, monitor, and assess the prerequisites, 
processes, and results of the education that Pegasus offers. The JEAs employ a 
specified set of evaluation criteria, all of which are clearly and explicitly related to 
the Standards and Guidelines for Quality Assurance in European Higher Education 
Area (ESG) (2015). In addition, each of the universities (or the responsible faculty) 
has the opportunity to add complementary criteria or questions to the evaluation 
matrix of the specific round of evaluation. The JEAs were piloted in 2016 and 
evaluated in 2017 before their more detailed design was finally decided upon. 

The JEA task force met on 6 occasions and organised 2 larger seminars with 
around 40 participants from the 3 HEIs. Three sub-groups were also formed, and 
they looked at possible clustering for academic subjects, administrative 
implementation, and internal and external communication. The JEAs were presented 
in 2015 at a national higher education conference in which questions of legitimacy 
and ways to mobilise staff support were emphasised. 

When introducing the JEA system at Pegasus, the vice chancellor emphasised 
that highlighting the ownership issue — i.e. that the HEIs themselves were initiating 
and implementing the JEAs and that it was not a system imposed “from above” via 
the SHEA — was important. Ownership was framed in emotional terms: as a 
“victory”. By framing the JEAs in this way, the management hoped to bring about 
acceptance and willingness to work with the JEAs and the overall IQA practices to 
which they belonged. Whereas the work within the JEAs was organised with 
collegial representation, the implementation of the JEAs appeared to run along 
rather-traditional paths from the top of the organisation down to the bottom; 
decisions were initiated and taken by the management, processed by the quality 
group, and then channelled to faculties that were “activated” (Pegasus 2015). 


Virgo 


Virgo is a small and rather young university college. According to the Virgo web- 
site, strategic specialisation and close cooperation with international and local com- 
panies and the public sector are its significant features. Research, it is claimed on 
the Virgo website, provides knowledge, innovations, ideas, and a qualified work- 
force in cooperation with external stakeholders. In addition, educational activities 
are organised to facilitate cooperation through mentorship, guest lectures, advisory 
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boards, and students’ written degree projects. Despite its specialisation, Virgo offers 
programmes and courses in subjects related to informatics, engineering, business, 
health, and education. This HEI is also organised as a line management system with 
a governing board, vice chancellor, and pro-vice chancellor. Virgo did well in the 
quality evaluations in 2011-2014. Apparently, this “success” has built organisational 
self-esteem and confidence that its study programmes, courses, and IQA system are 
of high quality. It has one faculty board and the dean is responsible for the IQA. The 
delegation of authority is built on the principle that the faculty board makes 
propositions that relate to systems of follow-up and evaluation and propositions for 
decisions regarding QA and quality development. 

The former IQA system at Virgo was described in a quality policy dated 2011, 
and this description was still on the website in 2016. In other words, Virgo was not 
in a hurry to reform its IQA system. Furthermore, it was concerned neither about 
cooperation nor with looking into how other HEIs built their IQA systems, which 
might be interpreted as manifestations of the above-mentioned confidence. However, 
during 2015, an inventory was performed for polices, directions, guidelines, and 
routines at all levels of the institution to make sure that they had in place a coherent 
system without any gaps. Besides the dean, the representatives for the disciplines 
and study programmes are important in developing a new IQA system. The faculty 
board arranged a couple of seminars in 2015 to act on identified absences; for 
example, one seminar was intended to make explicit the mandate and responsibility 
for leading roles — such as the ones responsible for study programmes, research 
groups, disciplines, and doctoral studies. In these seminars’ PowerPoint 
presentations, the success of Virgo in relation to all other Swedish HEIs in the last 
round of the national evaluations was highlighted. The faculty’s message to the 
responsible actors in the departments and programmes was to not settle down 
because “the hard part is to maintain high quality ... and that is why a systematic 
and continuous development- and QA work is called for” (Virgo 2015, emphasis 
added). 

Virgo was aiming at an IQA system that involved proactive and continuous fol- 
low-ups as close to educational practice as possible. The idea was to make eventual 
quality problems visible at an early stage to give support to correct or finishing 
courses or programmes. It was a system that was designed to permeate a cyclic 
process: forming a discipline, programme, or course; planning and carrying it 
through; continuously following it up (formative); conducting regular (summative) 
evaluations of it; and winding it up. This process had clearly defined roles and 
mandates, work models, and functions for coordination. It also comprised distinct 
routines, infrastructure, administration, support systems, key figures, management 
information support, channels for information and communication, competence 
provisions, and processes designed for organisational learning. It was a system that 
was designed to “build a culture of quality” by strengthening the cooperation 
between the academy and administration. It was aligned with the ESG and oriented 
towards national goals. It also contained alumni and programme inquiries, self- 
evaluations, and additional statistical figures. However, in the material, the risk of 
“unnecessary bureaucracy” was highlighted as an important challenge. 
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Orion 


Orion is a large, old, comprehensive university, with almost all faculties represented 
(medicine, law, social science, science, and theology). On its website, the university 
stresses its ambition to lead the world in research and education. Orion has a line 
management system with a governing board, a vice chancellor, a pro-vice chancellor, 
a small number of deputy vice chancellors, combined to a certain extent with a 
collegial management system. The internal quality work is assigned to a special 
central group that works with issues of evaluation and education quality, and this 
has been the case for many decades. The formal responsibility for decision-making 
concerning IQA is with the faculty boards. In the national quality evaluation of 
higher education 2011-2014, Orion did very well, as did most of the old and well- 
established universities (Ericson 2014). 

Also, representatives (active researchers/teachers) from all faculties and student 
representatives compose a central committee. The intent of the committee is to 
support faculties and departments in their work with quality and give advice to the 
vice chancellor in such matters. Orion had an elaborate programme that was based 
on academic values and virtues for internal work with quality issues in education, 
research, and extramural activities in place since 2002 and was revised in 2008. This 
programme responded to the university’s internal demands for quality and was 
combined with external demands for quality in education to emphasise accountability. 

Shortly after the government memorandum was presented in spring 2015, a task 
force comprising a chairperson, two representatives from the faculties, three student 
representatives, and three experts from Orion’s central administration, was selected 
by the vice chancellor. This group started to map activities already in existence in 
the faculties and tried to identify what was needed so that they could live up to the 
expectations expressed in the government memorandum. From the very beginning, 
the group was determined that Orion’s new IQA system needed to be minimalistic, 
allowing the faculties to add what they found important and develop slightly 
different models — depending on their variations of disciplinary and scientific 
traditions — and directions for the study programmes. The intent was for the new 
system to be as little of an inconvenience as possible but be adapted to the national 
system proposed in the government memorandum. 

The task force met continuously during 2015 and spent considerable time in 
presenting their work to internal groups — such as the vice chancellor and head 
management group, all faculty boards, heads of departments, and student unions. In 
so doing, their ambition was to collect ideas for improving the evolving design of 
Orion’s IQA system and provide an opportunity for criticism to be channelled. This 
process aimed at legitimising the new system and at making the implementation 
smooth. The task force faced the challenge of designing an IQA system that did not 
add requirements of collecting, documenting, and reporting information. A small 
international advisory group was formed for support and critical comments during 
this process (Orion 2016). The task force’s final proposal was circulated within the 
university during spring 2016. 
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Orion’s new IQA system includes evaluations of study programmes and courses 
that should be undertaken systematically at least every 6th year by “at a minimum 
two external assessors and at a minimum one colleague from a different faculty” 
(Orion 2016, p. 6). All evaluations within the new system should take the Higher 
Education Act, the Higher Education Ordinance, and the ESG as a starting point. 
The evaluation process includes a self-evaluation, external review from assessors 
and colleagues (peer review), and the involvement of teachers and students in the 
entire evaluation process, and it should lead to an evaluation report that includes the 
central results from the peers, the self-evaluation report, and suggestions for what 
measures need to be taken (Orion 2016). The faculty boards should summarise what 
needs to be done, be responsible for follow-ups within a year, and organise public 
access to all evaluations. 

The new system was met with overall positive reactions among staff. Two obser- 
vations are important. First, knowledge about and involvement in the new IQA sys- 
tem were not distributed within the whole university. Also, resources for the new 
QA system were considered a problem, even by the task force: 


Of course, there are many who say “how is this supposed to work? How much time are we 
supposed to put down on this?” (...) and that is perhaps people who do not think evaluations 
are that important, rather an unnecessary evil. (Orion faculty representative 2016) 


The vice chancellor awaited detailed demands from the SHEA before making the 
final decision on IQA system’s design. In the meantime, preparatory work was 
done: “Either way, the vice chancellor will most probably announce resources for a 
small number of pilot evaluations before that, as part of Orion’s preparations” 
(Central task force informant 2016). 


Hercules 


Hercules is one of Sweden’s specialised HEIs with university status. It is regarded 
as a distinguished and internationally recognised educational and research 
institution. It is a relatively small university with a university board that has overall 
responsibility. Special boards — similar to faculties — for education, research, and 
graduate education are at the level below this. The university board has an external 
chairperson and a vice chancellor, and the deans of the special boards are the top- 
level managers of the university. 

The board of education makes policy decisions regarding EQA for the university. 
A special unit is responsible for developing specific assessment instruments, and 
teachers and programme managers are responsible for collecting information 
required for the IQA system. Over the years, Hercules has done relatively well in 
quality evaluations but has also received criticism. 

Around 2010, Hercules developed a dedicated strategy and took action to 
strengthen its work with QA because one of the university’s major study programmes 
had failed in the national quality evaluations. The university internally mobilised the 
board, deans, and heads of the departments that were involved in the criticised 
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programme. Furthermore, an external quality expert was appointed with the mission 
of reviewing the university’s QA system, including the internal evaluations of 
undergraduate programmes conducted before the national quality evaluation. One 
recommendation was to develop a clearer system for QA that would strengthen the 
overall accountability structure. Prior to the introduction of the SHEA’s new model 
for the quality assessment of HEIs, the university’s infrastructure for IQA and 
concrete methods for evaluating study programmes and courses were also gradually 
tightened. To make the programme managers, heads of departments, teachers, and 
students aware of the evaluation forms to be used, the university established 
communication channels via the intranet, the established framework of decision- 
making. In addition, the IQA was further adapted to the ESG. Surveys and 
instructions for quality assessments are found in the indicators for assessing the 
ESG’s quality. Furthermore, training and curricula were deliberately and 
systematically adapted to the wording of the Higher Education Act and the Higher 
Education Ordinance. 

Hercules allocated resources to maintain and develop IQA, while work was also 
performed at several levels as part of the mission that the departments have no 
resources for completing. In efforts to develop and improve the IQA, Hercules 
cooperated with several other HEIs in a network. The network aims to share 
experiences and compare ideas and ways to implement QA without any binding 
joint decisions. 

Hercules’s IQA is a cyclic evaluation model that is based on national and internal 
learning outcomes, and it evaluates, documents, and thus clarifies information on 
processes and preconditions for goal attainment. It includes student surveys, specific 
evaluations, thematic evaluations, exit polls, alumni surveys, course evaluations, 
working-life surveys, and additional evaluations. 


Policy Enactments in the Reform Interval 


The case descriptions above have offered empirical illustrations of policy processes 
and practices during a reform interval in four HEIs. Earlier, we discussed the national 
and governmental policy work that was taking place at this point. Our study indi- 
cates that the enactment of EQA policy is rather lively at the national level, within 
the four HEIs and across the entire higher education sector. It is also clear that con- 
texts and conditions matter regarding how the four HEIs tackled this interval between 
two national EQA systems, resulting in a period of uncertainty and preparation that 
took on different forms across the studied cases. Processes of developing and revis- 
ing existing IQA policies and practices were initiated and developed differently in 
the four cases in relation to their respective QA histories — that is, both in relation to 
what they had done and in relation to how they had been assessed by the SHEA in 
the 2011-2014 national EQA system. Depending on their size, they also could use 
very different economic resources for QA-related activities. Orion has had a special 
unit for this purpose for decades, whereas Virgo — being a small and young institu- 
tion — had to rely on the dean tackling this job. We note that the four HEIs started 
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their IQA work at different times during the reform interval; some were more proac- 
tive and initiated internal work quite early on and did so before the government 
decision and even before formal intentions and written directions were formulated 
(Pegasus, Orion), and others waited a bit longer (Hercules, Virgo). 

This means that some of the difference in speed, intensity, and comprehensive- 
ness of the work with preparing IQA systems between our cases can be understood 
by relating those activities to the contexts and characteristics of the individual HEIs. 
We approached the analysed work of governing and of policy in term of enactment 
(Ball et al. 2012). We discerned that the enactments during the reform interval were 
numerous and varied, and each case had its own IQA story to tell. At the same time, 
there are apparent similarities across the analysed cases. Despite the uncertainty 
during the reform interval, HEIs tended to take (some form of) action: They have all 
developed IQA policies. They have been invited to participate in the state commission 
of inquiry to produce a new national system in a process of dialogue, consensus 
seeking, and decision-making. The HEIs have distributed tasks and responsibilities 
in the form of roles and task forces; have arranged meetings; have produced 
documents, including external and internal evaluations, policies, plans, directives, 
models, PowerPoint presentations, and memorandums; and have organised activities 
(seminars, hearings, mail dispatches, blogs, and other means of information) to 
anchor and get support and approval for their planned and evolving IQA systems, be 
it in various ways. 

The HEIs’ work with IQA design was clearly a top-down affair, and procedures 
were initiated and steered from the organisational centre to its peripheries (if it ever 
got that far). Top-level administrators and vice chancellors initiated and mobilised 
the work, and this was largely not a matter for staff on the ground. Another signifying 
trait in these processes is the perceived need and talk about dialogue and trust, as 
well as avoiding unnecessary bureaucracy. The rhetoric of enhancement (see 
Saunders 2014) has been embraced — quite explicitly in many cases — and this 
conceptualisation of quality and improvement is voiced and embraced in the HEI 
processes. In many ways, quality is perceived as a constant, ongoing, and non- 
separable part of everyday workings. However, somewhat paradoxically, and as we 
will return to below, processes and designs are firmly steered from the top down. 


Governing “Between” Reforms: Anticipation and Action 


In this final section, we will discuss the reform interval in terms of governing. One 
first observation is that to the extent that a national policy is “absent”, European 
input in the form of ESG functions as apparently naturalised sources of guidance. 
As one of our informants said: 


We meet each other. It is great. The last two meetings have been about what do you do and 
how do you handle this, have you started to adjust [to the forthcoming but not yet approved 
external demands, our note], how do you do it? We give each other advice. Some of us have 
reviewed how the present IQA systems comply with ESG and then shared this with the rest 
of us. And that is great. (Hercules, Faculty representative 1) 
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This informant voices a striking similarity across cases, namely, that there are 
important homogenising forces serving as a cue giver during this reform interval: 
the ESG. All HEIs not only have explicit and visible references to these standards 
but the standards are also explicitly communicated and disseminated externally 
between HEIs and within them. In November 2015, the agency SHEA published a 
translated version of the ESG in Swedish, and in our reading of the HEI policy 
documents, we clearly discern how this has moved into the HEIs’ vocabulary. On 
the one hand, the act of translation signifies the importance attributed to this 
document on a national level, but the informants in our data also seem familiar with 
the English version of the ESG and assigned them central roles in their internal IQA 
work before this translated version was published. 

HEIs are thus “embedded” within the European policy community, where the 
ENQA offers “norms and directives which require domestic compliance at the very 
least consideration” (Jacobsson et al. 2015, p. 2). It is also notable how actors within 
the higher education sector engage in information-seeking and brokering activities 
where they pick up messages provided by the political level or the SHEA. Thus, as 
noted by Jacobsson and colleagues (2015, p. 4) “micro-steering in the shape of 
‘steers’ and other subtle signals or even anticipated reactions among the civil 
servants allows the core executive to control the bureaucracy”. 


A Temporary Motor Failure in the Evaluation Machinery? 


We would also like to return to an idea introduced in the chapter “National Evaluation 
Systems” — the ideal typical notion of evolving “evaluation machinery” in higher 
education. There are a number of observations that we find interesting in this 
context. One of them regards “permanence”, which is one important characteristic 
of evaluation machines (Leeuw and Furubo 2008; Dahler-Larsen 2012). The 
findings presented in this chapter suggest that the institutionalisation of an evaluation 
machinery through, for instance, the promotion of evaluation culture within the HEI 
sector make the machinery sturdy in case of temporary motor failure. HEIs take on 
evaluative work as a form of decentralised spare engines that secure the permanence 
of the evaluative activities. Permanence during the interval is secured by other 
characteristics of the machinery. Notably, modes of organisational responsibility 
have been installed in HEI organisations and in the minds of certain key actors 
working within the evaluation field, that is, “evaluators” that are working within and 
between organisations rather than external to them. This group, which we already 
have introduced in this book as “qualocrats”, can be seen as the offspring of a 
“marriage between administration and evaluation” (Leeuw and Furubo 2008, 
p. 165) and is most likely an important reason that HEIs responses to the reform 
interval are so similar. This group of actors carries and possesses certain knowledge 
in and of EQA in higher education, which is brokered and promoted, and moves in 
the wake of meetings to initiate cooperation across HEIs, as well as to and from the 
policymaking arena, the ministry, and the political administration. 
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Understanding Governing in the Reform Interval: 
Standardisation and Homogenisation 


The enactments during the interval and the similarities that we have identified seem 
to evolve through such cooperation and exchange of experience of IQA work or 
what DiMaggio and Powell (1983) label normative isomorphism. Here, Virgo turns 
out to be an exception, as it is quite self-sufficient and exposes a sense of self- 
esteem due to their good turn out in the previous national quality evaluations, as we 
understand it. Another similar feature is the fairly top-down-oriented strategies all 
four HEIs use in how they have organised the work in developing their IQA systems. 
In a Swedish context, this is interesting because the so-called autonomy reform 
from 2010 gave way to a legal framework, meaning that the HEIs can organise their 
delegation of authority and formal internal decision-making as they please with the 
requirement that there are scientifically competent persons and students represented 
in these instances. In all our cases, the delegation of authority when concerning 
EQA rested with the faculty (scientifically competent instance). 

However, the IQA work was steered and governed from the centre and top. As far 
as we can tell, whether the preparations reached out to the departments and to the 
“street level” of the HEIs is questionable. This behaviour, where central management 
at the HEIs takes the helm, may be ascribed to the uncertainty felt during the reform 
interval, similarly to what DiMaggio and Powell (1983) call mimetic isomorphism. 
In times of uncertainty, this conceptual explanation suggests that central management 
overtakes responsibility from the more collegial instances to achieve some unified 
policy direction. We suggest that this is perhaps also a question of trust, where in 
Sweden trust in collegial responsibility within the HEIs seems to be presently 
decreasing (SOU 2015:92). This is also shown by the move in several HEIs to 
so-called line management systems (Sørensen et al. 2015, p. 6). Line management 
systems, where the decision-making power rests with formal leaders (not necessary 
scientifically trained) in the organisation, have evolved as a fashion in HEIs in 
Sweden (Sahlin and Wedlin 2008). Issues of quality in research and education, 
traditionally a responsibility for faculty and colleagues (researchers and teachers), 
are increasingly becoming the responsibility of managers (of an organisation and 
the economy) and quality and evaluation experts (Forsell and Ivarsson Westberg 
2016; Hall 2012). In the chapters “Re-launching National Evaluation and Quality 
Assurance: Governing by Piloting” and “Evaluation Machinery, Qualocrats, and the 
Seemingly Inevitable Problem of Expansion”, we return to the issue of an emerging 
cadre of “qualocrats” who are doing EQA work in Swedish higher education. 

There are also some striking similarities in our cases when it comes to how parts 
of the IQA systems are to be carried out, where the internationally common model 
of external reviewers? is to be used as well as that of so-called self-evaluations. It is 


? External reviewers may include peers (like in peer reviews, the collegial way of assessing the 
quality of scientific work). In the Swedish case, “external” refers to a HEI external group of peers, 
students, and representatives from other areas of society like employers. 
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hard to say if this is an imitation of an international discourse of evaluation in higher 
education, an imitation of earlier Swedish national systems, or an imitation of 
fellow HEIs in Sweden. It may well be more like a fashion, since this model is 
applied in so many HEIs globally. If the stress on the ESG — also visible in all our 
cases — is taken into account, Sweden and, as a result, the HEIs apparently feel 
coerced to accommodate to those in one way or another. 

Another similarity is the conflict between a wish to develop and design rigorous 
IQA systems and avoidance of an expansion of bureaucracy in terms of having to 
collect/produce and systematise more information, install more functions related to 
QA (e.g. QA officers at the faculty and department level), and constantly revise and 
follow-up QA activities and the entire system. This is by no means a new concern 
and has been observed in several public sectors in terms of an audit explosion 
(Power 1996), audit society (Power 1999), or evaluation society (Dahler-Larsen 
2012; see Ek 2012; Lindgren 2014 for Swedish examples). This conflict, and in our 
cases, a readily acknowledged concern, relates to our final observation. It has also 
been corroborated in our interviews: There is a general expansion of evaluative 
activities in what is developed and planned in the four HEIs. Thus, adding to the 
observations in the chapter “National Evaluation Systems” on the expansion of and 
evaluation in higher education in Sweden, this chapter has shown that the interval 
has not stalled the continuation of this overall growth process. 


Finally 


The issue of expansion is interesting from several perspectives. Even though 
increasing audit often includes control of control in terms of modes of self- 
evaluation, it is traditionally imposed on organisations from above or by external 
powers. This Swedish case illustrates a somewhat different development, where the 
policy interval has fuelled an uncompelled evaluation expansion within a “willing” 
higher education sector (c.f. Jacobsson and Nordström 2010). In other words, HEIs 
are not necessary victims of “evaluation rage”; rather, they seem to indulge in a kind 
of “evaluation gluttony” where enactments of EQA systems risk becoming too 
costly, too excessive in quality and quantity, too hasty, and too greedy.’ Apparently, 
HEIs — to a larger or lesser extent — see augmenting IQA systems as an important 
means of solving quality-based problems. They thus dedicate more time and energy, 
professional knowledge, and work to increasingly specialised roles and organisations 
of IQA. In the following chapters, we will continue to explore these issues as the 
reform interval comes to an end, and the 2016 national EQA system is developed 
(the chapter “Re-launching National Evaluation and Quality Assurance: Expectations 
and Preparations”) and implemented (the chapter “Re-launching National Evaluation 
and Quality Assurance: Governing by Piloting”). 


This type of gluttony is discussed among others by Aquinas (1265-1274). 
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Relaunching National Evaluation A 
and Quality Assurance: Expectations oy 
and Preparations 


Christina Segerholm 


Abstract In this chapter, we describe and analyse the design of the evaluation and 
quality assurance system decided by the Swedish Parliament in 2016. Particular 
stress is put on how one part of this system — the institutional reviews of the higher 
education institutions’ internal quality assurance processes — was set up by the 
Swedish Higher Education Authority (SHEA). The aim was to explore governing of 
higher education in the signals expressed through the design and requirements 
decided by the SHEA. The SHEA public reports, guidelines, criteria, and templates 
for the higher education institutions and evaluators, and interviews with SHEA staff 
were used for this purpose. The analysis shows that the Standards and Guidelines 
for Quality Assurance in the European Higher Education Area were important 
influences in the design work. It also shows that the design supports the governing 
by objectives and outcomes logic and a notion of “quality” in higher education as 
equivalent to work with internal quality assurance systems at higher education 
institutions. We argue that this design forms the basis for constitutive effects like an 
instrumental notion of higher education. 


Introduction 


Throughout the book, we have described and analysed the emergence of evaluation 
and quality assurance in the Swedish higher education system, with particular 
emphasis on the most recent developments. Here we will concentrate on the 2016 
system, which was preceded by the highly criticised 2011-2014 system, and later 
by a period without national evaluation and quality assurance (EQA) characterised 
by a multitude of activities and discussions throughout higher education. 
Commissioned to develop a new national system, the Swedish Higher Education 
Authority (SHEA) worked intensely to create a design for a new EQA system, first 
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from autumn 2013 to spring 2014, at which time the government decided to appoint 
a special investigator. Later, when the parliament decided on the new system from 
spring 2016, the SHEA intensified its design work. During this period of reform 
interval, higher education institutions were occupied with creating and improving 
internal quality assurance systems that should match requirements not yet decided, 
as described in the chapter “Enacting a National Reform Interval in Times of 
Uncertainty: Evaluation Gluttony Among the Willing”. 

To reiterate, in March 2016, a new national EQA system for higher education 
was decided by the parliament. Signals to the higher education institutions (HEI) of 
what parliamentary decision to expect had been disseminated by a government 
memorandum outlining the planned system. The SHEA had also worked to spread 
information and collect comments about what was thought would be decided by the 
parliament, particularly during autumn 2015, as the chapters “Hayek and the Red 
Tape: The Politics of Evaluation and Quality Assurance Reform — From Shortcut 
Governing to Policy Rerouting” and “Enacting a National Reform Interval in Times 
of Uncertainty: Evaluation Gluttony Among the Willing” show. The ramifications of 
the new system were finally decided in March 2016. The government then 
commissioned the SHEA to work out the design and details and how to best 
implement the system. What then, did this system, look like? 

In this chapter, we describe this 2016 national EQA system and pay particular 
attention to one component, institutional reviews of higher education institutions’ 
internal quality assurance (IQA) systems. Our aim is to analyse this national system 
as a matter of what it is meant to achieve (not what it actually achieves or how and 
what it influences), the motives, and the direction of the governing process (what is 
evaluated and how) as it appears in SHEA documentary materials and staffs’ 
experiences. We argue that the designs of national EQA systems direct attention to 
different parts of the HEIs’ activities and organisations, and as such, designs are 
also part of governing higher education. 

We used the following questions to organise our work with the chapter: 


e What is evaluated in the 2016 national evaluation and quality assurance system? 
Why? By whom? How? With what consequences in terms of expectations? 

e How is the work of designing the 2016 national evaluation and quality assurance 
system described by the SHEA actors? 

e What knowledge and ideas are valued and promoted within the 2016 system in 
general and in the institutional reviews in particular? 

e What are the implications for higher education governing? 


In the chapter “Governing by Evaluation: Setting the Scene”, we underscored an 
understanding of governing as a variety of deliberate processes and work performed 
by different actors, in different places through policies and various means and 
activities (Clarke 2015). Apart from describing the actual design of the 2016 national 
EQA system, this understanding also directs us to pay attention to the actors at the 
SHEA and their experiences in the design work. As this chapter in a way is a 
continuation of the retrospect in the chapter “National Evaluation Systems”, we 
once again make use of a view of higher education based on Hopman’s (2003, 2008) 
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idea of a move in management, from being an internally “ill-defined problem” 
controlled and evaluated by professionals (colleagues/peers) called “management 
by placement” to a “well-defined problem” managed, controlled, and evaluated by 
external “expertise” using indicators and standards, called “management by 
expectations”. Following Hopman’s argument, this move is connected to issues of 
accountability in institutionalised practices like education, where the boundaries are 
weak and changing over time, rendering them “ill-defined”. As education expands 
and its problems become even more “ill-defined”, accountability demands may be 
harnessed by a transfer to a more “well-defined” enterprise through the use of 
indicators, standards, and EQA systems, since “accountability needs something that 
can be counted, or where it is at least possible to measure the distance between 
expectations and results” (Hopman 2008, p. 425). Consequently, expectations in our 
text denote what is at the fore of the design of the national EQA system, in the 
expectations, ideas, and values it promotes. 

This understanding can be combined with what Dahler-Larsen (2012, 2014) 
analyses as “constitutive effects”. Constitutive effects are constitutive in the sense 
that evaluative activities like EQAs — including indicators, criteria, and guidelines — 
affect the phenomena, enterprises, or practises that are assured, assessed, or 
evaluated. Or, as Dahler-Larsen phrases it in relation to performance indicators: 
“The indicator helps define the concept it claims to measure” (Dahler-Larsen 2014, 
p. 975). This means that the designs of the national EQAs are in part defining what 
quality is in higher education, as well as signalling ideas of what higher education 
and HEIs are all about. 

The chapter rests on documentary materials like public reports from the SHEA, 
guidelines and templates for the institutional reviews as they were tested in a pilot 
and quite extensive interviews with nine SHEA staff in different positions during 
2015 and 2016 when the new national system was developed and implemented. 


The 2016 EQA System: Suspenders and Belt 


The government’s arguments for the new national evaluation and quality assurance 
system stressed the importance of high quality in higher education in order for 
Sweden to be competitive and secure future work opportunities (Ministry of 
Education 2015, p. 3). Large investments in higher education that have proven 
profitable are to be followed by continued expansion of the higher education system. 
This calls for persistent efforts to increase quality, it was said (ibid.) One objective 
was that: “(a)ll female and male students should know that they receive education 
of high quality” (ibid.). National quality assurance was put forward as the means to 
achieve this. These hopes were later formulated into objectives by the SHEA, stating 
that the planned system was both to control the performance of study programmes 
and to work for quality improvements (SHEA 2016a, p. 6, b, p. 6). This balance 
between a system that both controls and develops is hard to strike, as was testified 
by staff at the SHEA: 
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This is a role that we have, that these evaluations shall lead to development and the system 
shall be supportive, and at the same time we shall control because it is after all public funds 
that are used, and this ought to be controlled so they are used in the right way. It is a 
challenge. (SHEA staff 4, September 2016) 


One prominent idea with this EQA system is that the HEIs should take on a more 
active role in quality assurance through internal quality assurance systems (IQA), 
while the SHEA should be responsible for evaluating these internal systems. “They 
[the HEIs, our clarification] have the major responsibility for, or they have the 
responsibility for quality in higher education...” (SHEA staff PB6, November 
2015). Another idea of note is that the national system is to be cohesive so that it “is 
useful for all four components” (SHEA 2016b, p. 7; see below about components) 
of the system and also supports the HEIs’ internal quality assurance systems. An 
additional pronounced idea, and also an aim, is that the system should “not only 
have international legitimacy, but also ultimately contribute to a greater 
internationalisation of Swedish higher education” (SHEA 2016b, p. 5). This is done 
by adhering to the Standards and Guidelines for Quality Assurance in the European 
Higher Education Area (2015) or the ESG. When the entire system is in place and 
running, the SHEA once again plans to apply for membership in the European 
Association for Quality Assurance in Higher Education (ENQA) (SHEA 2016a, 
p. 12; SHEA 2016b, p. 12).' This is thought to enhance legitimacy for Swedish 
higher education abroad and facilitate student and faculty mobility. Throughout the 
SHEA’s development work with the national system, the ESG and ENQA 
membership was at the fore: 


...in order for us to live up to our ENQA membership, we have to do reviews of the HEIs’ 
internal quality assurance systems, and this is something that the sector itself and the 
politicians stress. (SHEA staff 1, September 2016) 


In the chapter “Europe in Sweden”, we concluded that the SHEA has been, and still 
is, a central brokering/dissemination organisation for European policy on higher 
education and quality assurance. This is even more evident in their development 
work and design of the latest EQA system. The importance of the European 
perspective and ENQA membership is present in the SHEA report to the government, 
where there are constant referrals to the ESG. 

Looking at other national EQA systems (the Nordic countries, England, Scotland, 
and the Netherlands) and relying on the internal knowledge and experience in the 
SHEA were important in developing of the system: 


And it is a strength that we have so much experience to get both from other countries, but 
also from our own experience. We, in fact, have now very much experience in the 
organisation of evaluations, at different levels. We have the old ones, we looked at previous 
systems, and we have looked at processes before. We have looked at outcomes before. Now 
we pick the best parts of our history. (SHEA staff 4, September 2016) 


! In February 2019, the SHEA announced that the process of applying for ENQA membership had 
begun (SHEA 2019). 
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Compared to the preceding national EQA system, the increase of what the HEIs 
must pay attention to is striking. As we presented in the chapter “Enacting a National 
Reform Interval in Times of Uncertainty: Evaluation Gluttony Among the Willing”, 
the EQA system is comprised of four components or types of evaluations: 

— Appraisal of applications for degree-awarding powers 

— Institutional reviews of the higher education institutions’ quality assurance 

processes 

- Programme evaluations 

- Thematic evaluations. (SHEA 2016b, p. 16) 

The first component assesses whether or not sufficient conditions exist in order 
for a HEI to award degrees and licenses. The second directs attention to how the 
HEIs’ internal quality assurance activities and organisation manage to “ensure” that 
all courses and programmes are good enough. The third type is the evaluation of 
quality in study programmes and focuses on outcomes and student attainment as 
they are laid down in the Higher Education Act, the Higher Education Ordinance, 
and System of Qualifications — that is — the expected learning outcomes for different 
degrees and licenses (SFS 1993:100, appendix 2). The fourth component, thematic 
evaluations, may include such areas as sustainable development and the HEIs’ work 
therein. 

The four SHEA evaluation types also entail reviewing and assessing four so 
called aspect areas. They are (a) “governance and organisation”; (b) “environment, 
resources, and area”; (c) “design, teaching/learning, and outcomes”; and (d) “fol- 
low-up, actions, and feedback” (SHEA 2016b, p. 17). The last aspect area is inte- 
grated into the other three aspect areas, and so its assessment criteria for that 
aspect area are included in those of the aspects. On top of that, the evaluations 
include three perspectives: student and doctoral student, working life, and gender 
equality (see Fig. 1). The SHEA claims that all these components, aspect areas, 
and perspectives are based on Swedish law and ordinances and the ESG (SHEA 
2016b, pp. 17-19). The gender equality perspective is a special assignment from 
the government and is something that “is close to their heart” (SHEA staff 3, 
September 2016). 

All evaluations in the national EQA system are carried out through the HEIs’ so 
called self-evaluations, external assessment panels with on-site visits and public 
reports with decisions and follow-ups. Different aspect areas are emphasised for 
each component evaluated (see Fig. 2). For example, all aspect areas are included in 
institutional reviews while in programme evaluations focus on aspect areas, 
“environment and resources” and “design, teaching/learning, and outcomes” (SHEA 
2016a, p. 21). Also included in the programme evaluations — and part of the basis 
for judgements — are students’ pre-graded degree projects (independent work 
papers). These are to be assessed by the external panels in order to judge whether or 
not the expected learning outcomes have been attained for different licenses and 
degrees (SHEA 2018a, p. 15). 

As can be seen, the EQA system encompasses several different types of evalua- 
tions that target different parts of the HEIs. This comprehensiveness of the system 
is something the informants at the SHEA also brought up as a possible problem: 
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National quality assurance 
system for higher education 


Saas 
Follow-up, actions and feedback 


Governance Environment, Design, teaching/ 
and organisation resources learning and 
and area outcomes 


Student and doctoral student perspective 


Working life perspective 


Gender equality perspective 


Developmental monitoring 


Fig. 1 Overview of components, aspect areas, and perspectives in the 2016 EQA system. (SHEA 
2016b, p. 18) 


National quality assurance 
system for higher education 


HIGHER EDUCATION ACT AND ORDINANCE AND ESG 


as: 
Follow-up, actions and feedback 


Governance Environment, Design, teaching/ 
and organisation resources learning and 
and area outcomes 


Developmental monitoring 


Fig. 2. Overview of what aspect areas and perspectives that are emphasised in the different evalu- 
ation types (components). (SHEA 2016b, p. 21) 
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... [have said several times that I think this [the national system, our clarification] maybe 
will be too resource demanding. We get a bit too little out of it in relation to all the resources 
we put into it. Perhaps we should have skipped some of these parts. (SHEA staff 2, 
September 2016) 


A general weakness in the whole system might be that we are now starting a giant system 
that includes, that is, I don’t think we’ve had this comprehensiveness in any other national 
system. (--) Can we and the higher education institutions cope with this? (SHEA staff 3, 
September 2016) 


As part of the design work and implementation process, several activities were car- 
ried out in which the evolving design was presented and feedback was collected in 
meetings with five reference groups that commented on the system as a whole and 
on the specific types of evaluations, i.e. the different perspectives, or evaluations 
directed at specific levels or education programmes, like teacher education (SHEA 
2016b, p. 49). More active work was also asked from the advisory group concerning 
the whole system and meetings with three advisory groups for specific levels or 
education programmes (ibid.). Furthermore, SHEA staff travelled around the coun- 
try and arranged several meetings at different HEIs, in which the national system 
was presented and discussions were held with representatives from the HEIs. This 
also gave the HEIs opportunities to compare their work with the development of 
their internal quality assurance systems (Observation notes 12 May 2016, SHEA 
presentation at Faculty of Social Sciences, Umea University). Information about the 
design process, information meetings, etc. was continuously displayed at the SHEA 
website and in their monthly newsletters. 

All four types of evaluations (components) were tested in small-scale pilots dur- 
ing 2016-2017. These pilots were also a planned effort to implement the 2016 EQA 
system; we portray the pilot of institutional reviews in the next chapter. Several 
meetings with HEIs and external assessment panels involved in this particular pilot 
were carried out, and in all these evaluation processes, SHEA staff had an important 
role: they acted as project and process leaders, steering the evaluation processes 
forward and arranging all kinds of practicalities, like schedules for interviews, site 
visits with HEIs, and meetings and coordination of different assessors’ written 
statements. According to the SHEA staff, it was crucial in their work to know about 
the Swedish higher education system, to have an academic degree in any subject, 
and to be socially competent and able to lead an external panel in their assessment 
work. They have opportunities to continuously develop their knowledge in 
evaluation — particularly about quality assurance — by attending international 
conferences, most often in the Nordic and European countries. 

In October 2016, the SHEA estimated the following numbers of the different 
types of evaluations during 2016-2022: approximately 20 appraisals of applications 
for degree-awarding powers, 46 institutional reviews, 650 programme evaluations, 
and 2-3 thematic evaluations (SHEA n.d.). 
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Designing evaluations for different purposes like control/accountability or develop- 
ment and capacity building is not easily done, since different purposes direct atten- 
tion to different parts of what is evaluated. The Swedish EQA system is meant to 
entail both control/accountability/compliance and support/development/improve- 
ment of quality, meaning it has to collect information of all sorts of preconditions, 
processes, and outcomes at several levels in the HEIs (see, e.g. Owen 2006 for dif- 
ferent types of evaluations for different purposes). Assessment criteria also have to 
be developed for each of these parts. The SHEA staff expressed some critical 
insights about the system’s comprehensiveness, both concerning their own work 
and on behalf of the HEIs. It is rather easy to imagine the enormous amount of time 
that has to be spent on collecting information, making sense of it in self-evaluations, 
assessment processes, writing reports, etc. Any given HEI is probably going to be 
involved in and have to respond to a number of these nationally required EQA pro- 
cesses, as well as carry out their internal system processes. Over the years, and 
through working with the requirements of different national EQA systems, the staff 
are likely to accumulate knowledge on EQA and the different meanings of quality 
in higher education. 

Designing the 2016 national EQA system, implementing it, and thus making it 
the national policy entail a lot of work: internal activities at the SHEA discussing 
different approaches and directions of the system, arranging meetings with reference 
and advisory groups and HEIs, visiting and learning about other countries’ national 
systems, continually up-dating and disseminating information, etc. 

What we now see is a national system that has parts of all the previous systems 
that were described in the chapter “National Evaluation Systems” or covers most 
ingredients of evaluations in higher education (Harvey 2010). It may be conceived 
as a sort of sedimentation/layering (Bleiklie and Kogan 2007, p. 482) of former 
ideas, expectations, experiences, activities, and knowledge. However, as Harvey 
and Newton point out, it is not easy to know exactly what the purpose(s) of the 
national system is, whether it is “the educational provider, or the specific programme, 
or the learner, or the output of the programme or the institution” that are the ultimate 
aim (Harvey and Newton 2004, p. 150); rather, it is all the evaluation objects that are 
targeted by the four components. Following Harvey and Newton (2004) and their 
analysis of external quality evaluations in higher education, it can be argued that the 
model of self-evaluation, external review, public report, and follow-up may not be 
the most appropriate for all these different purposes, intentions, directions, aspect 
areas, and perspectives. When the SHEA fully and energetically emphasises and 
incorporates the ESG, this also means that the EQA system indirectly promotes a 
behaviouristic view of learning, as the ESG concerned with learning, teaching, and 
assessment (Standards and Guidelines for Quality Assurance in the European 
Higher Education Area 2015, p. 12) is based on and sustains a behaviourist 
epistemology with predefined end behaviour and outcomes, as shown by Murtonen 
et al. (2017). Apparently, this is also something that goes hand-in-hand with the 
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implementation of the Bologna Process (Murtonen et al. 2017, pp. 114-117). 
Several research studies have shown that such an epistemology is a restricted 
theoretical basis for education, teaching, and learning because these processes also 
include cognitive and social processes that are more open-ended as to what the 
knowledge outcomes may be (Murtonen et al. 2017). As a complimentary 
understanding of learning, Marton et al. (2008) offer a viewpoint based on 
phenomenography in order to achieve higher order learning, holistic understanding, 
and deep learning in higher education. 

With these remarks about the overall 2016 EQA system, we move to the compo- 
nent institutional reviews of the HEIs’ internal quality assurance systems. The rea- 
son we concentrate on this evaluation type is that it is the most comprehensive one; 
this type was also underscored in policy texts, arguing that more responsibility for 
quality in higher education should rest on the higher education institutions 
themselves. 


The Institutional Reviews: Quality as Assuring Quality 
Assurance 


In this section, we describe the design of the institutional reviews of HEIs’ internal 
quality assurance systems as it was set up for the pilot of this type of evaluations. Pilots 
were performed as part of the implementation process of the new national system. 

In the Swedish language guidelines, the purpose of the institutional reviews is 
stated as: 


to control that the HEIs’ internal quality assurance work ensures high quality in courses and 
programmes, and to contribute to the HEIs’ quality development. (SHEA 2016c, p. 8, our 
italics) 


Time and resources were also spent on translating these guidelines into English in 
order to facilitate international understanding or make recruitment of foreign 
assessors easier, as we understand it. The meaning is slightly different in the English 
translation and is an example of how the meaning is somewhat changed in the 
processes of translation. In our view, the Swedish version expresses a harsher tone, 
compared to the English version, reading as follows: 


to examine whether these processes ensure high quality courses and programmes and to 
support the HED’s quality improvement efforts. (SHEA 2016d, p. 8, our italics) 


There is also another purpose, linked to European policy. Again, it has to do with 
Sweden’s — or to be more precise, the SHEA’s — membership status in the ENQA. In 
order to be eligible according to ENQA requirements, the applicant has to live up to 
all ESG. The HEIs’ internal quality systems are a cornerstone of these requirements 
that must be reviewed by an external evaluation agency, as described in the chapter 
“Europe in Sweden”. With reference to these ENQA requirements, one SHEA staff 
said: “We can’t say that Sweden should have its own sort, (---) So therefore, the new 
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ESG have been very central. We have worked a lot with them when it comes to this 
component, the institutional reviews” (SHEA staff 2, September 2016). 

Other ambitions with the institutional reviews are to push the HEIs “a step fur- 
ther” and to have them ask questions like: 


...but how do you know that you have control? What do you have, how can you verify it and 
how do you use the information you get? What measures do you take if you find things that 
work a bit poor, and at the same time how can you spread good examples? (SHEA staff 1, 
September 2016) 


The institutional reviews are said to be based on the Higher Education Act, the 
Higher Education Ordinance, the System for Qualifications, and the ESG. The 
importance of the ESG was repeatedly stressed by the SHEA staff: 


We worked a lot in relation to the ESG when it comes to the institutional reviews. So, the 
ESG have become a very, (---) that is, the ESG have been revised during this period, so there 
were new ESG. [We] translated them into Swedish. It is the ESG in English that are valid, 
but we did a Swedish translation and put down quite a lot of resources into it. (SHEA staff 
2, September 2016) 


These institutional reviews are aimed at assessing how well the HEIs’ internal qual- 
ity systems contribute to “ensure and improve the quality of courses and pro- 
grammes at all education cycles, and covers all aspect areas and perspectives” 
(SHEA 2016d, p. 9). Furthermore, the reviewers assess how systematic, effective, 
and proactive the HEIs’ internal quality systems are, as well as their integration in 
the HEIs’ organisation and the activities they have. It is clear from the guidelines for 
the institutional reviews that such characteristics are necessary in order to be judged 
adequate. 

The aspect areas and perspectives in the institutional reviews are the same as the 
ones listed above, and every aspect area is in turn divided into a number of narrower 
aspects. These different aspects and the three perspectives are each assessed based 
on criteria developed for that particular aspect or perspective (SHEA 2016c, p. 9). 
Criteria for the different aspects closely follow the ESG; even so, some modifications 
are visible due to the national context and how to translate and interpret the ESG so 
as to fit Swedish legislation and academic tradition. 

There are different types of materials that are part of the reviews and on which 
the judgements are made, including descriptive statistics from the SHEA, a so-called 
self-evaluation report by the HEIs, a student report, web interviews, site visits, and 
areas of focus (SHEA 2016d, pp. 9-12). The self-evaluation report should be no 
longer than 50 pages, written in a set template developed by the SHEA and organised 
around the main ESG (Part 1) within each aspect area (see Fig. 3). 

As an example, Part 1 of the ESG is directed at “Student-centred learning, teach- 
ing and assessment” (Standards and Guidelines for Quality Assurance in the 
European Higher Education Area (ESG) 2015, p. 12). One of the guidelines for that 
standard reads: “The assessment allows students to demonstrate the extent to which 
the intended learning outcomes have been achieved” (ibid.). In the guidelines for the 
pilot of the institutional reviews, this is operationalised into the assessment criteria: 
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cont. Aspect area: Design, teaching/learning and outcomes 


3.2 Aspect: Goal attainment 


Assessment A. The HEI ensures that its courses and programmes are 

criteria designed, developed and implemented with a clear connection 
between national and local goals, teaching activities and 
examinations. The HEI ensures that all students have good 
potential for achieving their established goals within the planned 
time. 


B. The HEI works systematically to follow up and evaluate whether 
students’ actual learning outcomes correspond with the expected 
learning outcomes. Planned or implemented measures following 
such a review are communicated to the relevant stakeholders. 


Guidelinesfor Briefly describe and then evaluate the selected work methods 

HEls for meeting the assessment criteria and analyse the strengths 
and areas for improvement. It is good to illustrate the report 
with specific examples of how the systematic quality assurance 
procedures are evident in and contribute to improving the 
operation. In addition, include the student and doctoral student 
perspective, employer and labour market perspective and gender 
equality perspective when relevant. 


Fig. 3 Illustration of try-out guidelines translated into English by the SHEA. (SHEA 2016d, 
p. 25) 


The HEI works systematically to follow up and evaluate whether students’ actual learning 
outcomes correspond with the expected learning outcomes. Planned or implemented 
measures following such a review are communicated to the relevant stakeholders. (SHEA 
2016d, p. 25)? 


The HEIs should assess how well their internal quality assurance system fulfils the 
requirements set in the guidelines for all aspects and perspectives (SHEA 2016d, 
p. 25). The student report is meant to give students a chance to express their views 
on five pages at a maximum. The SHEA materials, the self-evaluations, and the 
student reports should be uploaded to a particular electronic system that the SHEA 
has developed for this purpose, called UKA Direkt (ibid.). 

In the description of the pilot of institutional reviews, web-based interviews are 
to be conducted approximately 5 weeks after the self-evaluations have been 
uploaded in UKA Direkt. One purpose of these web interviews is to clarify issues 
raised from the self-evaluations and to prepare areas of focus (SHEA 2016d, 
pp. 11-12) chosen by the external panel in order to check whether or not claims in 
the self-evaluation reports are really practised. For example, the panel could check 
if course evaluations are systematically performed and their results communicated 
to relevant stakeholders. Areas of focus are checked in the site visit, mainly by 


?This wording is not dramatically changed in the revised guidelines from 2018 (SHEA 2018b, 
p. 14). 
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interviews with head management, academic leaders, faculty, and students and/or 
doctoral students (ibid.). 

The number of assessors in the external assessment panel for the institutional 
reviews is not specified, in the pilot guidelines (SHEA 2016d, pp. 12-13). However, 
other requirements on the external panel include at least one assessor with 
international experience and one assessor with expertise in the Swedish higher 
education system. One assessor should be a student representative, and another 
must be an employer and labour market representative (ibid.). Furthermore, the 
instructions in the guidelines list what the assessors’ assignments as: 

- Discussing the assessment of aspect areas, aspects, perspectives, and assessment 
criteria 

— Participating in meetings during the entire evaluation process 

- Representation at the initial meeting with the HEIs to be included in the evaluation 

- Reviewing the various assessment criteria, justifying the reviews in writing, and 
specifying what data the reviews are based on 

— Jointly preparing questions for interviews with HEI, student, and any employer and 
labour market representatives that the HEI cooperates with 

— Summarising the assessments in a joint statement including the assessment panel’s 
joint judgement and proposed decision (SHEA 2016d, p. 13) 

The HEIs are asked to nominate assessors to the external assessment panels, but 
the SHEA has the final say. It is sometimes hard to compose a panel from the 
nominations, meaning that the SHEA staff have to recruit assessors: 


but it was, it was just two out of four who I picked from the nomination list (---) one asses- 
sor who was in the old, that is, when we reviewed the HEIs’ internal quality systems in an 
earlier cycle 2008 or 2009. (Hawke project leader SHEA,* March 2017) 


Once recruited, the assessors should partake in a short training session (1—2 days in 
the pilot) to inform them of this particular type of evaluation, the ESG, and, most 
importantly, to ensure they have a common understanding of the criteria and that the 
review is carried out as it is supposed to be carried out: “(t)heir assignment and role 
should be clear, and they should understand, have a common picture of the whole” 
(SHEA staff Utb, March 2017). They are also informed about the electronic device 
on which they communicate within the panel, the UKA Bedémarvy. According to 
the SHEA staff, useful knowledge, competences, and/or experiences for the external 
assessors are that they “have worked at a more central level at their higher education 
institutions” (SHEA staff 1, September 2016), such as vice chancellors or deans; 
some international experience and experience in evaluations and working in groups 
are valuable. The guidelines urge thorough knowledge on the Swedish higher 
education system and quality assurance (SHEA 2016d, p. 13). 

All aspects and perspectives should be assessed in relation to their specific crite- 
ria, and the instructions to the external panels specify that these “must be judged as 
satisfactory for the overall assessment to be positive’ (SHEA 2016d, p. 13). 
Furthermore, the external panel should: 


3“Hawke” refers to the HEI that was under review. Other HEIs in our materials on institutional 
reviews were Eagle and Falcon. 
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...focus on the results of the quality assurance procedure — that is — that it systematically 
and effectively ensures and improves the courses and programmes. In addition, an 
assessment will be conducted of how well the HEI’s quality assurance procedure 
systematically identifies strengths and ensures they are maintained and developed, as well 
as how the areas for improvement are identified, followed up and addressed. In this context, 
the SHEA would like to emphasise that it is considered positive for the quality assurance 
procedure to be able to identify and manage deviations and areas for improvement. How 
relevant stakeholders are given the results of the quality assurance procedure is also to be 
assessed. (SHEA 2016d, p. 13) 


A preliminary report from the external assessment panel should be sent to the HEIs 
under review for them to correct and comment on any factual errors. A final report 
is the basis for the SHEA decision, with the rating in the pilot on a two-point scale: 
approved or not approved (SHEA 2016d, p. 14). For HEIs that are not judged 
as approved, there are follow-ups within a year with reviews of the improvements 
deemed necessary by an external panel. The HEIs should then show what measures 
they have taken to come to correct these weaknesses. Experiences and good 
examples are also to be communicated in feedback conferences and dialogue with 
the HEIs under review independent of the judgements (SHEA 2016d, p. 14). 

The SHEA staff who worked to develop the new EQA system are strong believ- 
ers in evaluation and quality assurance. They expect that the institutional reviews 
will lead to “...the development of faculty competences — that they learn” (Eagle 
project leader SHEA, March 2017), or that “on a more overarching level, they [the 
HEIs, our clarification] will have to shape up (---) and hopefully this seeps down to 
the courses and programmes so they become better” (SHEA staff 5, March 2017). 
Other expectations concern the possibility that the institutional reviews enhance 
students’ perspective and make the HEIs better prepare the students for working 
life, so that higher education is useful and the reviews ensure students’ rights to 
good education. There are also some more cautious thoughts about what these 
reviews may do to the HEIs, albeit not many: 


We must not forget that the choices we make when it comes to focus and our basis, become 
very normative for the higher education institutions’ continued quality assurance work, and 
this is something that has to be handled responsibly. (------- ) 

Now when we do the institutional reviews, and focus is on talking about the ESG, that 
the higher education institutions should do their own follow-ups and evaluation. Of course, 
it is clear, that there will be a lot of focus on that now. (SHEA staff 1, September 2016) 


Comments on the Design Work of the Institutional Reviews 


The institutional reviews are aimed at assessing HEIs’ internal quality assurance 
systems, and these have to live up to the ESG as is also expressed in the overall 2016 
national EQA system. We have demonstrated the ambitious work the SHEA laid 
down in translating the ESG into Swedish and align the institutional reviews with 
them. Here, the influx of European quality assurance policy becomes or is transduced 
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into actual design work and work with guidelines, templates, instructions, and 
training. 

The SHEA also translated the institutional review guidelines into English. 
Translations from one language to another always pose some difficulties when it 
comes to expressing the meaning of the words. The citations from the two guidelines 
show that the term “control” (kontroll in Swedish) is translated to “examine” in the 
English language guidelines, and “contribute” to “development” in the Swedish 
version (bidra till kvalitetsutveckling in Swedish) is translated to “support 
improvements” in the English version. Our point is that policy and governing work 
always include translations of different kinds; these shifts in meaning may be more 
in tune with the ESG, as they are expressed in the English language. However, 
another type of translation is also visible in the design and implementation process 
of the institutional reviews: translating the ESG into criteria that are meaningful in 
a Swedish higher education institution context. 

Notable in the institutional reviews are the rather strong political ambitions to get 
the HEIs to work more intensely on gender equality issues. A stress on HEIs to 
become more market oriented is visible in the requirement of representatives from 
that perspective in the external assessment panels and in this particular perspective 
in the guidelines. Similar changes in higher education have been reported from 
other countries in several studies (e.g. Massen and Stensaker 2011; Schuetze et al. 
2012). Student influence is also stressed and goes hand in hand with an increased 
ambition to pay attention to students’ rights, as a matter of individualised learning. 

The behaviouristic understanding of education and learning is also evident in the 
institutional reviews, as the ESGs are so conspicuously used as a basis for judgement. 
We note that judgements and the process of assessment in these reviews are based 
on a theory of analytic assessment where separate judgements are made on each of 
the pre-set criteria and then aggregated. The final judgement is “then built up from 
a series of small-scale decisions. When the steps are followed systematically, the 
grade follows as a logic outcome” (Sadler 2009, p. 161). According to Sadler (2009), 
such assessments may be rather inadequate for making judgements based on more 
open-ended and compiled materials, such as those used in the self-evaluation report 
and interview answers. 

Central to quality assurance would be what meaning “quality” is given in the 
design, in the guidelines, and in the actual processes. Our interpretation of the 
design and the guidelines of the institutional reviews is that quality in higher 
education is fundamentally constructed as equivalent to the existence of internal 
quality assurance systems at the HEIs, which should assure high quality. Whether or 
not such systems actually do so is not really possible to know, since “high quality” 
is not defined. On the other hand, our analysis shows that the design and guidelines 
signal and underline the goal-/objective- and outcomes-/results-oriented rationale of 
governing; a gender equality perspective; a labour market perspective; and a student 
influence perspective. This view of quality in higher education is only slightly 
compatible with what the vice chancellors expressed as quality in higher education 
in the chapter “Navigating Higher Education Institutions in Times of Quality 
Assurance — The Assumptive Worlds of Vice Chancellors”. It is perhaps a conscious 
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decision by the SHEA to leave the main part of handling the “What is quality?” 
question to the HEIs. It is also in line with what is commonly referred to as 
“academic freedom” (Polanyi 1951, in Hartl 2012, p. 310) or faculty power over the 
scientific content and assessment of quality, in research and higher education. 
However, the design of the institutional reviews requires the HEIs to set up internal 
quality assurance systems that match the criteria of all aspects and perspectives, 
circumscribing such power and power to decide how the HEIs’ internal resources 
are best used to achieve good education and research practice. 


Discussion 


The 2016 EQA system’s overall design and work, as well as the institutional reviews, 
are based on the ESG. This means that the European higher education and quality 
assurance policy now is fully incorporated in the Swedish higher education policy 
context. It still remains to be seen whether or not this system is considered by the 
ENQA to be independent enough in relation to the government. Throughout the 
design process, the SHEA worked diligently to align the national system and insti- 
tutional reviews with the ESG. This work entailed interpretations and translations of 
different kinds and in Freeman’s words: “to translate is to represent in a new form 
something previously or otherwise represented differently in another language or 
medium” (Freeman 2006, p. 1). We have shown that shifts in meaning exist in lan- 
guage translations but are used in order to make the ESG acceptable in the context 
of Swedish higher education context. This is in part how European policy travels and 
is made acceptable in different nations (e.g. Grek and Lawn 2012). 

We interpret the design to show some particular Swedish values, including politi- 
cal stress on gender equality, the labour market, and student influence. Although not 
absent in global or European education policy, the refinement of those perspectives 
is related to increased political pressure and expectations of the entire Swedish edu- 
cation system to work more intensely in these directions. 

The SHEA staff developed aspect areas, criteria, guidelines, etc. for the 2016 
system. They worked with the ESG and how to help them function in the Swedish 
context. They worked with the incorporation of political demands on higher 
education (like gender equality and labour market perspectives) and constant 
dissemination of information. In this work, the SHEA also spent time and energy to 
consult several advisory and reference groups, as well as arranging discussions with 
stakeholders. These efforts lead to a comprehensive national EQA system further 
supported by the component institutional reviews. With the SHEA’s clever 
requirements of the HEIs to take part in national EQA processes and to develop 
internal quality assurance systems, external and internal scrutiny merges and has the 
potential to reach far into educational practice. Hence, consequences of the 
institutional reviews will be that the HEIs are expected to live up to this 
comprehensiveness, which demands resources: not only must they develop and add 
to existing IQA systems; they must also organise internal work to respond to the 
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requested activities of the other types of evaluations administered by the SHEA. For 
example, the programme evaluations judge their ability to meet national standards 
of goal attainment set by the external panels. Expectations of what knowledge, 
skills, and values to include in programmes and courses are thereby communicated 
to faculty and students, as well as how students’ degree papers should be assessed, 
albeit indirectly. This is yet another signal and possible constitutive effect (Dahler 
Larsen 2012, 2014) that sustains the objective-/goal- and results-/outcomes-oriented 
rationale of governing higher education. It also represents a particular view of 
education, teaching, and learning that may influence teachers’, students’, and 
researchers’ understanding of what it means to be a teacher or a student and the 
meaning of higher education and knowledge generation as processes. According to 
this rationale, education and research are conceived as entirely predictable, while 
learning and new scientific insights most often are open-ended and unpredictable. In 
Hopman’s (2008) words, the design of the 2016 EQA system turns higher education 
into a “well-defined” problem, managed by external “expertise” using indicators 
and standards that direct what is expected from the HEIs. 

In our description and analysis of the design of the 2016 EQA system, we have 
observed that the meaning of “quality” in higher education largely seems to rest on 
the existence of extensive internal quality assurance systems at the HEIs. Based on 
this fact, another possible constitutive effect may be that such a conception of 
quality becomes more widespread in the future (Dahler Larsen 2012, 2014). 


Finally 


The transformation of national EQA policy for higher education into a concrete 
design of a national system is part of the governing work at the state level, as we 
have shown in the description of the SHEA’s work in setting up the 2016 EQA 
system. More everyday national EQA work also demands activities. Pilots were 
conducted to fine-tune the design of all four components and as an implementation 
strategy. What happened when the design was put into “action”, so to speak? In the 
next chapter, we explore this by following the pilot of institutional reviews. 
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Relaunching National Evaluation A 
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Joakim Lindgren, Linda Rönnberg, Agneta Hult, and Christina Segerholm 


Abstract This chapter seeks to explore and discuss enactments in the process of 
piloting a part of the 2016 national evaluation and quality assurance system, namely, 
the pilot of institutional reviews in which higher education institutions’ internal 
quality assurance systems were evaluated by the Swedish Higher Education 
Authority. The chapter analyses the work and experiences of the different actors that 
took part in these processes. It shows that the pilot includes extensive work that 
links people, places, policies, practices, and power in particular ways and that 
numerous translations are made at different stages and by different actors. The 
results highlight the amount and forms of work done in these processes in general 
and in particular by actors who we have labelled “qualocrats”. Their embodied form 
of expertise is mobilised as they move between and across different domains to 
enact and promote certain knowledge in and of evaluation and quality assurance. 
The chapter finally suggests that the deliberate temporal design as a pilot study 
opened up for mutual adjustments, learning, and dialogue but also gave rise to 
contradictory anticipatory governing signals. 


Introduction 


In this chapter, we turn to the implementation of the intentions of the 2016 national 
evaluation and quality assurance system that was analysed in the previous chapter. 
More specifically, this chapter focuses on a particular part of the national evaluation 
and quality assurance (EQA) system, namely, the Swedish Higher Education 
Authority’s (SHEA) institutional review of higher education institutions’ (HEIs) 
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internal quality assurance systems and processes (IQA). This form of external 
national evaluation was piloted on a small scale to test and develop the institutional 
review before its full-scale implementation, which was carried out during 2017 and 
included a handful of HEIs. We examined this pilot study at two of these HEIs (the 
Falcon and the Eagle) as the institutional review unfolded. 

This chapter aims to explore and discuss enactments in the 2016 EQA pilot pro- 
cess of so-called institutional reviews in which HEIs’ IQA systems were evaluated. 
We do this by analysing different actors’ work and experiences. Returning to our 
theoretical understanding of governing as a verb, and thereby “doings”, and our 
interest in exploring the relationships between governing, evaluation, and knowl- 
edge, these questions serve as guides: 


e What enactments did the pilot entail, and what actors were involved? 

e What kind of knowledge was mobilised and used in these enactments? 

e How can we understand the pilot as governing in relation to the higher education 
institutions taking part in the pilot and in relation to the higher education sector 
as a whole? 


We did about 30 interviews with actors working at the SHEA in external assessor 
panels and at different levels at the assessed HEIs. We did initial and follow-up 
interviews with these actors to capture experiences and opinions as close to the 
unfolding of events in the pilot as possible. We asked questions about thoughts and 
experiences from the different stages in these processes, as well as more general 
questions about perceptions of quality and EQA, to contextualise and situate the 
responses. We also collected and analysed a range of documentary materials, such 
as schedules and plans for the pilot study, assessment panels’ reports, and the pilot 
decisions by the SHEA. Another crucial artefact in the pilot was the self-evaluations 
from the HEIs. A self-evaluation is an instrument that literally materialises core 
aspects of governing through knowledge. The act of writing involves a transformation 
of knowledge from the diffused and abstract forms that it may take in actions, 
practices, and peoples’ minds into (supposedly) more concrete textual forms. Like 
all the other methods in the pilot portfolio, self-evaluation is not a direct evaluation 
of first-order activities; it is a mode of “control of control” (Power 1994, p. 15) that 
renders quality visible to external observers through inscription. However, the 
production of self-evaluations does not only make quality observable, it may also 
consolidate processes of meaning making; self-evaluations may “construct and 
define quality itself’ (Power 1994, p. 293). 

In particular, we studied the review of two HEIs: the Falcon and the Eagle. These 
HEIs are neither full universities nor extremely small and/or specialised, and this 
was our main reason for focusing on these HEIs for a more in-depth study of the 
work, experiences, and governing in the SHEA pilot. 

The rest of the chapter has the following structure: next, a description of the 
institutional review process offers a brief chronological overview of the process in 
the pilot. These stages, starting with preparatory work and ending in post-decision 
actions and reactions, are dealt with in more detail and with a particular focus on 
“governing work” (cf. Clarke 2015). We do this by analysing different actors’ 
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activities, including what they were doing and what they experienced, throughout 
the different stages. After this, the chapter situates the pilot in relation to our 
analytical frame by revisiting the initial three guiding questions. 


The Sequencing of the Review 


Looking at the SHEA’s work, extensive preparation work for the pilot was carried 
out by the agency — not in the least by designing the process for the pilot (see the 
chapter “Relaunching National Evaluation and Quality Assurance: Expectations 
and Preparations”) or by developing guidelines, templates, and materials for exter- 
nal assessor panels’ training, as well as by developing electronic software for shar- 
ing and storing information. Meetings with HEIs and others to disseminate 
information about the new EQA system and the design of the activity studied here — 
that is, the institutional reviews of HEIs’ IQA system — were also carried out. The 
HEIs that took part in the pilot volunteered to do so, which is important to recognise 
as one premise for the entire process. Another premise for the participating HEIs 
was set by the SHEA; this premise stated that any HEIs that would pass the pilot 
review and that were approved did not need to be reassessed in the full scale and 
mandatory cycle, while those that were not approved had to be reviewed again at a 
later date as a part of the mandatory cycle. 


The Sequencing of the Pilot 


The pilot started with an initial information meeting organised by the SHEA that 
targeted representatives from the handful of HEIs included in the pilot and the 
chairmen of the external assessor panels (one panel for each HEI). The SHEA 
staff — that is, the project leaders for the individual HEIs taking part in the pilot — 
were also at this meeting. The next step in the review process was the production of 
the HEIs’ self-evaluation reports to be submitted to the SHEA approximately 
2.5 months after the initial start-up meeting. About a month after receiving the self- 
evaluation reports, the external assessment panels, assisted by the SHEA project 
leaders, conducted web-based interviews with the HEIs. About 1.5 months after the 
web-based interviews, site visits took place at the HEIs. Members of the external 
assessment panels were accompanied and assisted by the SHEA project leaders 
during this process. Over a period of 2 months, the panels worked on finalising their 
reports. The finalisation of the report also included a meeting, during which all 
assessors met with the SHEA project leaders to discuss the different assessments 
with an aim of comparability between the panels in the review. Ten months after the 
initial information meeting, preliminary reports from the external assessor panels 
were sent to the HEIs for comments (called “sharing”’), meaning that the HEIs were 
provided an opportunity to correct factual errors and comment on the reports. The 
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reports were then finalised, and the SHEA published their decisions, which included 
the final report by the external assessment panels. 


Preparing to Assess and Preparing to Be Assessed 


In the following section, we examine different stages in the process outlined above 
more closely by highlighting how central groups of actors in this pilot study 
(HEI staff, assessors, and SHEA staff) initially enacted the different stages of the 
institutional review by turning to their preparatory work. 


HEIs 


As noted above, participation in the pilot was voluntary. The decision to take part, 
however, was based on different justifications in the two HEIs. Falcon management, 
who had an IQA system in operation, said that “we were ready [mature] for audit” 
(Falcon Vice Chancellor). Key actors within the management had been involved in 
national policy discussions, knowledge exchanges, and preparations of the new 
national system, such as in a referential group within the SHEA and the quality 
group of the Association of Swedish Higher Education Institutions (ASHEI, in 
Swedish SUHF), which organises vice chancellors and top-level HEI management.! 
The experiences managers at the Falcon had participating in such higher education 
networks provided insights into the new national system in general and into the 
institutional pilot review in particular. As a result, the invitation to take part in the 
pilot was enthusiastically accepted at the Falcon based on the conviction that this 
pilot could stimulate the improvement of an already existing IQA. 

The Eagle joined the pilot on rather different grounds. Participation was justified 
on the basis that it could offer external help in setting up and developing their IQA: 
“we may not be ready, but we will learn a lot on the way” (Eagle Vice Chancellor). 
Efforts at the Eagle to set up the new IQA were paralleled with no less than three 
other external evaluations in 2017. This engagement in evaluative practices is 
emblamatic for the zeitgeist and something that the forthcoming account of the pilot 
will provide many examples of. 

After being accepted for inclusion, the two HEIs immediately initiated extensive 
efforts to organise the internal work in terms of roles and tasks. First, the review 
process had to be thoroughly scheduled and coordinated in alignment with the 
overall SHEA schedule. Each HEI carried out inventories to find out what the 
organisation knew about itself. For example, they had to identify who in the 


! Among other things, ASHEI works to disseminate knowledge of and in the HEI sector, for 
instance, to develop common policy directions for HEIs and by policy advocacy aimed at the 
Ministry of Education and other policymaking bodies. 
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organisation carried specific knowledge about particular organisational areas and 
processes and what internal documents could make that knowledge visible. They 
had to identify key actors within the organisation who could provide such knowledge 
as a part of the self-evaluations and/or answer questions about the IQA in the 
upcoming interviews. Soon after, the HEIs also attended an information meeting 
with the chair of the assessment panel; the meeting was organised by the SHEA. The 
contact with the chairpersons was expressed by representatives at the Eagle as being 
soothing and comforting, since the approach — that is, to assess how the HEI can 
make sense of the way in which their IQA covers their needs — was perceived as fair. 

The work on the self-evaluations gave rise to particular challenges. Although the 
Falcon and the Eagle QA management both knew their weaknesses fairly well 
before the review, the interpretation of the SHEA’s intentions demanded collective 
efforts. The HEIs’ self-evaluations had to be organised according to a SHEA 
template. The basic challenge was to meet the SHEA’s expectations, which were 
perceived as rather vaguely formulated in the guidelines. The self-evaluations were 
crafted in a successive and collective process of interpretation and translation. The 
Falcon’s internal time schedule for the self-evaluation, visualised in a detailed 
spreadsheet, displays how the work with the self-evaluation was initiated in 
December 2016 and how it was followed by 27 activities, including identifying 
coordinators and reference groups, interpreting guidelines, holding numerous 
meetings, collecting data, holding discussions, undergoing multiple peer review 
processes, doing editorial work, doing proof reading, ensuring formal finalisation, 
submitting, and completing self-evaluation of the work. 

Writing a self-evaluation is not only a laborious matter of describing your own 
strengths and weaknesses; it is a puzzle to find out what the assessor may “want” 
and is thereby a source for queries among those involved: “What do you think they 
[the SHEA] want us to write here?” (Eagle Quality Management Staff 1). As noted 
by Falcon management, responding to items in a template to be submitted for 
external review is not an easy process of representation. On the contrary, it fuels 
insecurity. The number of instances that the HEIs could nevertheless “misunderstand” 
a given explanation of a ground for judgement by the SHEA is numerous: 


... the range of possible interpretations are endless (...): “well, it could be like this”, “well, 
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it could just as well be like this”, “yes, but do they mean process or?”. So, we were occupied 
up until the last day with trying to define [it]. And in the end, we just had to make up our 
minds and say, “no, this is what they mean”. (Falcon Quality Management Staff 1) 


The HEIs also discovered that they had to design images that would visually 
display their IQA systems. These images alone invoked collaborative achievement. 
As it turned out, the self-evaluations contained a number of detailed, complex, and 
multicoloured organisational models that served to illustrate aims, flows, visions, 
schemes, activities, differentiation, hierarchies, processes, and cycles. Overall, the 
basic ideas communicated in the self-evaluations drew on ideas from Total Quality 
Management (TQM) characterised by an emphasis on management responsibility 
for quality improvement, systematic and continuous analysis, and the improvement 
of work processes conducted throughout the organisation through means of involve- 
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ment and empowerment. Explicit reference to the so-called Deming Cycle (i.e. the 
wheel of continuous improvement stemming back to the evolution of engineering 
and industrial production in the 1930s) displays how such modernist schemes and 
modes of thinking are repeatedly repacked and moulded into HEI governance 
regimes (cf. Stensaker 2007). 

The self-evaluations also had to show that the HEIs worked with all four aspects 
(governance and organisation; environment, resources, and area; design, teaching/ 
learning, and outcomes; and follow-up, actions, and feedback) and three perspec- 
tives (working life, student influence, and gender equity; see the chapter 
“Relaunching National Evaluation and Quality Assurance: Expectations and 
Preparations”). This was to be corroborated by documents describing how this was 
achieved. Hence, according to the HEI informants, a lot of time was spent revising 
documents and uploading them on the SHEA’s web-based administrative system 
(UKA Direkt) because it was “important to clarify internal processes so they could 
be evaluated” (Eagle Teaching Staff 1). To illustrate the scope of this work in pre- 
paring for the review, the Falcon submitted no less than 77 supplementary docu- 
ments to “exemplify”, “demonstrate”, and “emphasise” different dimensions of 
their IQA (Falcon, self-evaluation document). Examples of such documents include 
the following: 


e Internal rules 

e Handbooks 

e Self-evaluations 

e Audits and reports 

e Development plans 

e Annual reports 

e Expert statements 

e Visions and annual reports 
e Matrixes and syllabi 

e Study guides 

e Examinations 

e Course evaluations 

e Seminar instructions 

e Schedules 

e Benchmarking reports 
e Guides 

e Routines 

e Information material 

e Application forms 


This body of documentation, and the institutional logic that their accumulation 
and employment constitute within the pilot, corresponds to Michael Power’s (2013) 
conceptualisation of audit trails. These “involve the routine production of artefacts 
which document work routines, but which are also the micro-manifestation of larger 
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performance regimes shaped by institutional demands for accountability” (Power 
2013, p. 1). According to Power (2013), the mediating function of this form of 
documentation “means that audit trails are definable as: evidential pathways which 
connect traces of micro-routines to performance reporting regimes and institutional 
environments” (ibid.). 

Thus, this early phase involved collective interpretation and translation, as well 
as tedious work that was strenuous and time consuming. As noted by one key actor 
at the Falcon, pressure to produce the self-evaluation led to extraordinary working 
conditions: 


We were up [late], working after midnight, the last several weeks. The last time I did such 
a thing was when I was a doctoral student, a very long time ago. I mean, a group of people 
drinking coffee after midnight.... (Falcon Quality Management Staff 2) 


This picture of a tight gathering of HEI actors taking a short break from work in 
the middle of the cold, dark Swedish winter night will be kept in mind when the 
reactions to the formal SHEA decision are described later in this chapter. The mid- 
night coffee break serves as an image of the substantial personal and collective 
investment that a review process of this kind can entail. Despite such demanding 
conditions, the actors did not emphasise the heavy workload or the time and 
resources required as particularly problematic — rather, the tight deadline for the 
self-evaluation that was set by the SHEA produced frustration and discontent at 
both HEIs. As one of our HEI informants said, there was not enough time between 
the initial meeting of 1 December 2016 and the deadline for submitting the self- 
evaluation of 24 February 2017 to sufficiently anchor the self-evaluation or to work 
and reflect in decentralised groups within the organisation: 


Within this short time span, we really had to compromise. We would have liked to do much 
more thorough, solid, and inclusive organisational processes. We did the best we could; we 
were out [in the organisation] and talked and collected documents and did such things (...) 
but it could have been done more extensively. (Falcon Vice Chancellor) 


Two observations deserve to be highlighted in this context. The first is that the 
Falcon found the focus of the review model to be too rigid. It is based on summative 
ideas on evaluation rather than formative ones (Scriven 1967). As a result, there is 
not enough space within the boundaries of the model to qualitatively produce, 
develop, and use local organisational knowledge as a means for improvement. 
Secondly, it is noteworthy — once again — how the enterprise of quality management 
and IQA work appears to trigger instincts and eagerness to do more and more and 
to add and incorporate evaluative activities to an expanding IQA. The SHEA is not 
the one pushing HEIs into further expansion and immersion; it is particular catego- 
ries of staff within HEI management, what Jacobsson and Nordström (2010, p. 178) 
would call “a community of the willing”. 
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The SHEA 


Let us initially describe the actors at the SHEA in the context of this particular pilot. 
One SHEA member of staff was assigned to each institutional review and named 
project leader. In addition, other SHEA employees were involved in designing and 
preparing the new national framework in general and the institutional review in the 
pilot in particular. These actors were also activated during different stages of the 
review. The SHEA actors’ backgrounds show many similar features and experiences. 
They had often worked at the agency for many years and held an academic degree, 
most commonly a Ph.D. Many of them also had previous experience from academic 
work and leadership. Extensive experience from EQA was also common. The 
SHEA project leaders were continuously and closely following “their” respective 
review by informing, organising, and interacting with review participants. In this 
way, the project leaders, to some extent, came to represent the agency in the eyes of 
the HEIs and the assessors. 

Within the SHEA, this first phase of the pilot consisted of two basic tasks: imple- 
menting the review model and organising the actual pilot process. A handful of 
HEIs initially volunteered to be reviewed in the pilot. The SHEA strived for HEI 
variation with regards to size, scope, and geographical location. Four HEIs was 
considered to be enough to fulfil this. The SHEA project leaders organised external 
assessment panels using strategic invitations to higher education actors and based 
on the names nominated by the HEIs. In addition to a chairperson, the assessment 
panel also included a non-Swedish assessor, an actor from the HEI sector, a working 
life representative, and a student. According to the SHEA informants, knowledge 
and experience from earlier work at the agency, HEIs, or similar settings were 
considered to be particularly valuable for assessment panel members — similar to the 
experiences and knowledge the SHEA project leaders possessed. In addition, 
leadership skills and abilities to organise processes and instruct participants in the 
review were considered to be important. 

Moreover, the SHEA project leaders administered the previously mentioned 
web-based platform where review material could be shared among parties, as well 
as a section on the web-based platform where the assessors in the external panels 
could communicate and share their work. The project leaders were also responsible 
for ensuring uniform and standardised processes across the different panels. This 
task also entailed making sure that the assessors understood the method that was to 
be employed in the review and that the review was consistent with principles of legal 
certainty. Thus, on the one hand, emphasis during the panels work was on a kind of 
sector-specific and contextual knowledge, such as from previous work in HEIs. On 
the other hand, such forms of embodied knowledge seemed to exist in parallel with 
ideals about objectivity based on formal principals of justice, where all cases must 
be treated equally (cf. Molander 2016, p. 32). We will return to this inconsistency 
when discussing the epistemic dimension of the pilot later in the chapter. 

Extensive preparations were conducted to design templates that were meant to 
serve as guides in the review. In addition, the SHEA produced training and guidance 
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material for assessors, which they used during a full-day training session organised 
for all assessors. The training of assessors foremost aimed at standardising grounds 
for judgements and assessments across the panels and members to make their future 
judgements trustworthy and legitimate. In our interviews, they repeatedly 
underscored the importance of dialogue (presumably a strategy to avoid the 
emotionally loaded policy processes preceding the existing national system 
described in earlier chapters in this volume). This dialogue orientation was 
manifested in the form of continuous meetings with assessors and with the HEI 
representatives and in the organised training of assessors. 


Assessors 


The SHEA requirements and strategic recruitment of external assessors resulted in 
panels with members who had the desired backgrounds. Most of them had substantial 
experience of evaluations in higher education, both as being evaluated in their 
department, discipline, educational programme, or institution and as assessors of 
earlier and former agency evaluations. Some of them took part in the reference 
groups that SHEA organised to develop the new 2016 EQA system. As a result, the 
assessors possessed knowledge about the system and its intentions, which they 
brought to the panel and the process. They also had great experience in academic 
work and management at different levels and were involved in QA at their 
department/institution. When assessors described what they need to know to take on 
their task, they emphasised experience with top management work at HEIs, previous 
experience from the SHEA evaluations, and knowledge about QA. As noted by the 
Falcon chairperson, their panel also benefited from the fact that the working life 
representative had specific and vast experience and knowledge about quality 
technology; it was a person “who knew the essence of the craft” (Falcon Assessment 
Panel Chair 1). The informants argued that the panels also benefitted from student 
and employer perspectives, which the representatives from these groups brought to 
the panels. 

The chairpersons (not the full panel) took part in an initial meeting at the SHEA 
and with HEI actors in December 2017. The panels then received documents and 
guidelines from the SHEA, which they reviewed. A second meeting at the SHEA 
(with all panel members) aimed at presenting information to and training the 
assessors by outlining the preconditions of the pilot, the process, the key indicators, 
and the components. During these initial meetings, a set of conceptual and practical 
questions were raised by the assessors. As we will see later, these questions — which 
were advanced at the beginning of the process — were to become imperative during 
the whole pilot. The questions concerned, for instance, how to weigh judgements, 
the role of the three perspectives (working life, student influence, and gender 
equity), details in the SHEA guidelines, and questions that concerned central 
concepts, such as the meaning of the word “to assure” [att säkerställa, in Swedish]. 
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Hence, the assessors had an opportunity to meet each other, with the SHEA and the 
HEI actors. 

At this stage, the assessors did not have a clear picture of the HEIs that they were 
reviewing or the roles and tasks that they were expected to perform. During the first 
meeting, the HEIs presented their organisations and IQA systems. Since the Falcon 
and the Eagle were perceived as deviating from “traditional modes” of how to 
organise HEIs, it invoked challenges for the assessors. Initial queries about who at 
the HEIs were suitable for taking part in the first web-based interviews at the 
respective HEIs were also made. The Falcon chairperson said that this panel used 
the first meeting to understand the particular organisational features of the HEI that 
they were reviewing: “We had to translate in order to understand. ‘What does this 
role mean?’ ‘Is this the same as head of department?’” (Falcon Assessment Panel 
Chair 1). This is another example of the particular translation problem that arises 
when the review implies a norm against which the object of evaluation is measured. 
These translations affect not only the actors under review (as exemplified in the 
problems described above in terms of writing a self-evaluation on the basis of a 
template) but the assessors who had to assimilate new knowledge about previously 
unknown organisational structures to conduct the review. 


Collecting and Providing Data 


The next stage of the pilot involved additional activities in which the assessors 
accessed data — uploaded by the HEIs — from the web-based platform. For the 
former group, deliberations and struggles about what to assess and with what criteria 
became increasingly pertinent. The panels used the HEIs’ self-evaluations as a basis 
for forming an initial impression and then organised their work and prepared for the 
upcoming web-based interviews. Each panel member produced an individual 
assessment with suggestions for questions the HEI representatives would be asked, 
and these individual panel member texts were then edited into one document by the 
SHEA project leader. Each assessment panel then held a meeting to decide which 
questions to ask the HEI representatives in the web-based interviews, in what 
interview each set of questions should be raised, and to whom they should be 
directed. This also meant that some questions were laid aside for the upcoming site 
visit. The assessors who voiced certain preferences about informants for the 
interviews requested informants who represented different subject areas and units 
and worked on quality issues. At the Falcon, where management of external 
evaluation was more developed, a “liaison central” was organised to prepare, 
organise, and follow up on different activities during the pilot. Among other things, 
this included the web interviews and organising post-interview deliberations. 
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Web-Based Interviews 


The web interviews were organised as 1-day sessions with scheduled focus group 
interviews with approximately 20 HEI actors in total. At the Falcon, the web-based 
interviews were not perceived as satisfactory. The format appears to have produced 
distance and certain difficulties. First, some of the students and teachers who were 
interviewed found it a bit daunting to have a group of top management professionals 
and professors asking “evasive semi-bureaucratic questions” (Falcon Student 1). 
The interviewees at the Falcon also felt they were being “put to the test” in a situation 
characterised by obvious power asymmetries and in which assessors were perceived 
to have certain normative preconceptions. These emotional reactions are partly 
linked to one of the interviews, where the Falcon actors claim that they were told to 
prepare for certain topics and questions, and then the assessors asked questions 
about other aspects. These actors felt that these questions were difficult to answer, 
as they did not target the actors’ roles in the organisation. In this situation, the earlier 
mentioned “liaison central” served an important function: to comfort interviewees 
in what can be described as a kind of “emotional debriefing”. The actors at the Eagle 
were, however, not as critical. Here, the assessment panel’s questions were perceived 
as being connected to the previously submitted documents, which fostered a 
situation where the interviewees were constituted as being capable and 
well-informed. 

We would like to draw attention to two issues. First, interviews taking place in a 
context of accountability are framed by stressors and anxieties. Regardless of 
whether the assessors manage to produce a safe situation, the actors representing 
their HEI entered the interview with the expectation of being able to answer 
questions that do justice to their specific organisation. Still, such systems are 
sophisticated, and whereas some knowledge is easily recapitulated, other knowledge 
often remains unformulated and is thus difficult to communicate. The preparations 
of the HEIs were characterised by the actors’ eagerness to be honest and 
knowledgeable about the organisation, as well as its strengths and weaknesses. In 
this context, the HEIs’ preparations before interviews served to inform these actors 
about their own work and organisation to pass this information on to the assessors. 
The second aspect that we would like to point out is the perceived problems in terms 
of fuzziness about what is assessed and how. For example, the HEI actors found it 
difficult to determine where the cut score was from the panel’s questions. For the 
assessors, the self-evaluation was a central document that served as a point of 
departure for what to focus on and what to ask about in the web-based interviews. 
During these interviews, the assessment panel members took turns interviewing and 
taking notes, as well as deciding if the information provided was satisfactory. The 
Eagle assessment panel specifically found the purposes of the web interview 
occasion and the site visit to be a bit unclear and struggled when it came to separating 
issues for the web interview and the later on-site visit. 
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Site Visits 


Generally, our informants describe on-site meetings as far more positive and pro- 
ductive than web interviews. These meetings were seen as a central activity for 
developing mutual understandings and exchanges for all involved parties. Thus, 
face-to-face meetings seem to bring dimensions of embodiment into the interview 
that encourages discussions that move the review process forward. The HEI actors 
described the encounters as nice conversations and the panel as interested and well 
prepared. However, on-site visits also implied additional work because they were 
not limited to the actual meetings between assessors and HEI actors; they required 
planning, organisation, and preparations, and they led to forms of post-production. 
Overall, the exercises and the work undertaken before and after the site visit point 
to the fact that, even though the pilot can be divided into different phases and 
activities for analytical purposes, all of these activities were closely interlinked as a 
continuous process. 

The SHEA project leaders continued to have a less visible but important role. 
They organised the site visit schedules, supported the panel chairperson, and kept 
time and discussions on track. The site visits were organised as 2 full-day sessions 
consisting mainly of 45-min focus group interviews with approximately 40-50 HEI 
actors in total, including management, teachers, students, doctoral students, and 
other staff and working life representatives. Each HEI actor involved in the site visit 
received a memorandum from the SHEA project leader outlining the visit’s schedule. 
The memorandum also described the pilot’s rationale, the assessment panel’s role, 
and the time schedule. It declared, among other things, that each HEI actor was 
expected to bring a nameplate because there would be no time for presenting 
individuals or the organisations and units they represented. There were 15-min 
breaks scheduled between each session, allowing the assessors time to prepare and 
the HEI actors a chance to switch places. 

The site visits were prepared on the basis of the self-evaluations and the SHEA 
guidelines, and they were specifically directed to the “audit trails’ (SHEA 2016, 
p. 12) of areas identified by the assessors. To qualify their understanding, the 
assessors prepared the interviews and requested additional materials from HEIs. 
The QA management at the Eagle experienced a new workload peak as documents 
had to be uploaded to the platform all over again, this time divided differently from 
the uploads undertaken in connection with the self-evaluation submission. They 
also had to produce new documents that retrospectively described decisions already 
made. In all, “it was like putting together half a new self-evaluation” (Eagle Quality 
Management Staff 1). 

In both HEIs, the site visit was prepared by specific pre-meetings for those 
selected for an interview with the assessors. Post-meetings were also held during 
site visits to transfer experiences from the initial interviewees to colleagues in line 
for upcoming interviews. As was the case when preparing for the web interviews, 
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these measures also increased people’s awareness about their own IQA to facilitate 
“thick descriptions” during interviews. This proved to be a successful strategy 
because one purpose of the site visits appeared to be controlling whether or not the 
HEI actors were grounded in the presentations given in the self-evaluations. In other 
words, site visits secured evidence that dialogue and enhancement had spread 
organisational knowledge within the organisation. One SHAE project leader 
explained: 


One important matter is that they have an IQA [system] that is inscribed in some kind of 
policy document. Then, we come and visit them with our basis for judgements and aspects, 
and we look at their policy documents and their systems [IQA systems], and we do our 
interviews with a whole range of people from different programmes and departments, 
levels, management from the top to the bottom, and students and doctoral students. Then we 
(...) examine whether it corresponds to what they have written in their own policy 
documents. Here, it is especially important that they have a dialogue. Most of the people we 
talked to knew about the system; in this dialogue-based system where they constantly talk 
to each other about what they do, they have a process and discuss quality issues. They think 
about how they can improve. It was not perfect, and there was someone who did not know 
[about the IQA], but if you think about all the people we talked to during the site visit... But 
the assessment panel felt that “this seems to work, they appear to keep track, they know 
their system, they talk to each other, management is in control over departments and 
programmes”. Enough people were aware and knew the system to motivate a Pass grade. It 
could have been better, but it was sufficiently many. If I try to quantify (...) if we say that 
we spoke to 80 people, and 65 of them kept track of the system, this is just an approximation, 
since you are asking about a level, just to get a theoretical number. If it had been only 20 of 
the 80 people we talked to, well then, it might not have been possible to give them a Pass 
grade. (Falcon project leader SHEA) 


Here we see that quality enhancement ideas are framed within a wider logic of 
accountability. One of the most important facets of quality enhancement, and what 
the institutional review seeks to measure, is that HEIs have an IQA system that is 
not just put on paper but is also actually a concern of the entire organisation. The 
exploration of the pilot’s next phase will show that formal assessments of such 
organisational processes are challenging. 


Reaching, Communicating, and Receiving Judgements 


Assessments involve complex processes of professional discretion and judgement. 
Instruments and frameworks in the form of rules and guidelines do not guarantee 
determinacy (Evans 2010). Thus, judgments in the pilot required individual and 
collective work and deliberation. In this process, the primary sources of judgement 
were, once again, the self-evaluations, their attachments and uploaded documents, 
and the SHEA guidelines. In addition, the site visits were also important sources of 
data to verify and clarify different areas. 
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The SHEA 


The SHEA project leaders assembled each assessors’ individual judgements and 
texts into a single document to facilitate panels’ communication. This was also done 
to prepare for a forthcoming meeting where review panels were invited by the 
SHEA to discuss assessments and judgements across the different panels. 
Furthermore, the SHEA project leaders supervised the panels in their judgement 
processes. For instance, project leaders asked assessors to specify what forms of 
evidence they could use to support their judgements. The project leaders emphasised 
that assessors’ overall judgements must be unanimous and comparable between 
assessment panels. They were also responsible for “calibrating” the individual 
decisions for each HEI in the pilot. Preliminary assessments were processed by the 
panels, officially endorsed by the agency, and later communicated to the HEIs. The 
HEIs, in turn, were allowed to comment on the content for accuracy. The report then 
went back to the panels, and they eventually submitted their final report to the 
SHEA. At this final stage of the review process, several internal QA activities took 
place within the agency itself and involved different actors, units, and levels of the 
agency. Finally, the director general sanctioned the decisions to be published. 


Assessors 


The two assessment panels experienced different intricacy levels. The Falcon panel 
started sensing that they were looking at a system that already met the criteria in the 
beginning of the process. As the IQA was perceived as functioning well overall, the 
actual work of making final judgements and writing the report was viewed as 
unproblematic. Still, some difficulties arose in deciding the “limits for pass or fail”. 
The Eagle panel had more difficulties. The Eagle IQA was quite new, and some 
parts were not operational and therefore difficult to assess. Within this panel, there 
were early discussions about relative weighting between criteria and about difficul- 
ties with how to interpret and assess the three perspectives. For example, insecurity 
emerged about what it really means that the IQA system “should be proactive, it 
should be systematic, it should be integrated and so on” (Eagle Assessment Panel 
Chair) and how this criterion ought to be operationalised. The panel members also 
discovered that the criteria were not internally calibrated. The criteria for gender 
integration, for instance, were so ambitious that they were perceived as almost 
impossible to pass. The assessment work went on throughout the entire review pro- 
cess in the form of consecutive discussions within the group. During the process, the 
SHEA project leader “emphasises, or she is particularly driven by, for example, the 
student perspective and also pushed that quite hard” (Eagle Assessment Panel mem- 
ber). The Eagle panel member argued that this weakened the relative independence 
of the assessors’ work, thereby adding further entanglement. As the deadline 
approached, there were still diverging opinions in the external panel. At this stage, 
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the SHEA project leader insisted on a unified and unanimous judgement from the 
panel. A consensus in the panel was considered necessary, and diverging opinions 
were to be transformed into a unified decision accepted by all panel members. After 
some deliberation, this was finally settled by formulating the overall judgement in a 
positive way while pointing to the areas that could not be judged as sufficient. The 
chair and project leader then worked with internal calibration and revision and care- 
fully edited the final adjustments of critical formulations. 

The method for summarising the judgements as outlined in the SHEA guidelines 
stated that all four aspects and subareas and all three perspectives must be approved 
for a final passing grade. If any of these failed to meet the required standards, the 
HEI must be included in a new review in the full-scale implementation. This method 
of summarising judgements eventually made it clear that the Eagle did not pass, in 
contrast to what some panel members had suggested. Because the guidelines did not 
allow for weighting between the subsections, i.e. a weakness in one subsection 
could not be “compensated” by strong performances in another subsection, the IQA 
as a whole could not be approved and given a passing grade in this case. 

The main goal in the assessment process, just like in the Falcon panel, was to 
check for a feedback loop in the QA work at all levels, i.e. to check that quality 
enhancement was sufficiently established at all levels and in all parts of the 
organisation in a spirit of engagement and responsibility. In one SHEA decision, for 
example, it is argued that: 


The site visit and the web interviews unambiguously show that substantial engagement and 
broad responsibility are taken for QA in daily work among staff and among students. 
However, the assessors got the impression that participation could have been even more 
widespread, especially when it comes to the category staff and students. (...) During the 
interviews, it was shown that many teachers and students did not know about the novice 
student survey or the improvement measures it resulted in. This is an area that the HEI 
needs to work on. 


Once again, this is a vision of a complicated system continuously engineered to 
secure improvement. Each specialised role or function within it has extensive 
knowledge about the overall system’s design, including its engineering as well as 
the effects of each component. Such ideas differ dramatically with the Hayekian 
ideals (Hayek 1945) presented in the chapter “Hayek and the Red Tape: The Politics 
of Evaluation and Quality Assurance Reform — From Shortcut Governing to Policy 
Rerouting” on how the governing of complex human systems ought to be decentral- 
ised and evolve on the basis of dispersed and tacit forms of knowledge. 


HEIs 


For the HEIs, the time between site visits and before the preliminary and definitive 
reports was perceived as long. When the preliminary judgements were communicated 
in the autumn of 2017, they evoked mixed emotions. The Falcon’s IQA was approved 
and got a passing grade. The management and the board were relieved by the 
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outcome, which they argued gave the organisation the positive affirmation they had 
hoped for when they initially signed up for the pilot. At the same time, they were a 
bit confused and even upset as they identified a lack of understanding of their IQA 
as it was interpreted by the assessors. The key actors at the Falcon had discussed 
whether or not to address these perceived problems in their comments to the 
preliminary report. These careful considerations indicate that the process was seen 
as risky by actors who worked within the HEIs and wanted their organisations on 
the safe side of a passing grade. 

Among the key actors at the Eagle were various expectations for the assessment’s 
outcome; some of the Eagle staff believed it would be a pass, and some believed it 
would fail. As it turned out, the Eagle did not pass, resulting in some disappointment 
and irritation. For example, some critical remarks were made on the assessors’ sen- 
sitivity to what came across as one single critique from students. Moreover, the 
preliminary outline of the SHEA report was perceived as hard to understand, and it 
was not correct on all points. Key actors at the Eagle commented on those errors in 
their formal reply to the preliminary report. They had hoped for revisions in the final 
decision, but, instead, corrections were attached to the end of the report. In other 
words, the perception was that the HEIs’ comments were not adhered to. 


Post-decision Actions and Reactions 


After the pilot decisions had been made public, the SHEA organised two meetings 
where the pilot was discussed and evaluated. A meeting with the HEIs that took part 
in the pilot took place first, and a month later, in January 2018, a larger so-called 
dialogue meeting took place with other HEIs that was to be reviewed in the regular 
upcoming cycle. The SHEA also conducted surveys with the HEIs involved in the 
pilot to map their experiences with the process. The SHEA worked intensely to 
refine the method on the basis of the pilot and the input from HEIs. In parallel with 
these institutional reviews, ongoing discussions were organised by the ASHEI. At 
four conferences between 2015 and 2017, representatives from HEIs shared their 
experiences by presenting and discussing their IQA systems. At an ASHEI 
conference in June 2017 (after the site visits), the HEIs that took part in the pilot 
presented their specific experiences and their suggestions to improve the new 
institutional review model. Key actors from the SHEA also attended these 
conferences, provided input, and gave updated information about their work. First, 
these activities display that the development of the institutional review was not 
limited to the pilot. Secondly, an analysis of the range of HEI and IQA presentations 
during the ASHEI conferences reveals a prevailing isomorphism in terms of IQA 
designs, as shown in the chapter “Enacting a National Reform Interval in Times of 
Uncertainty: Evaluation Gluttony Among the Willing”. 

In January 2018, the institutional review to be implemented in the regular cycle 
was presented at a dialogue meeting. The grades had been changed to three levels 
instead of two: pass, pass with reservation, and fail. The web interviews were 
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replaced by an additional site visit. There were also new guidelines with reduced 
details, new templates for self-evaluations, and student appendices that were 
published on the SHEA website. More specific requirements concerning assessors 
were added, such as a requirement of experience with gender equality work (SHEA 
2018a, p. 25). The short time frames for the HEIs to deliver materials were 
questioned, but the SHEA explained that “three weeks resulted in so much materials, 
it was the wrong signal. It should be exclusive materials” (SHEA staff, observation 
notes). The new model was presented at another meeting in March of 2018, with 
assessors and representatives from the four HEIs and student unions included in the 
first regular cycle. In an interview on the SHEA website, the SHEA coordinator 
notes that: 


There were not that many questions during the meeting, which suggests that the HEIs have 
followed the process of developing the method and are well-informed about the content in 
the new guide that describes how the review operates. From the response at the meeting, we 
can see that our dialogue meetings have contributed to the HEIs being well-informed and 
have a grip of the method and the process. (SHEA 2018b) 


Thus, the pilot developed and tested the institutional review, and it worked as a 
governing tool that informed and legitimised the new model within HEIs. As we 
will show in the following, however, governing signals from the pilot were not 
unequivocal. 


HEIs 


Contrary to previous national evaluations, this pilot did not receive much attention — 
neither media attention nor broad attention within each HEI. Moreover, the pilot 
seemed to largely confirm what the HEIs already knew. After receiving the decisions, 
the responsible staff at the Eagle worked with the identified weaknesses, most of 
which they were already aware of and prepared for, to improve their IQA before the 
upcoming reassessment in the ordinary cycle. It was seen as an efficient and safe 
way to improve and learn, to seek inspiration, and to align their IQA with other 
HEIs. Extensive work was directed to the assessed perspectives where the Eagle 
failed to achieve a “full score”, i.e. gender equality and student influence. Actors 
from the Eagle also sought to make sure that course evaluation follow-ups and 
feedback to students were improved, and they worked to clarify internal documents 
and processes for colleagues. Internal documents were also revised to more clearly 
accommodate a student perspective and to — as it was said — make texts more “alive”. 
Bureaucratic textual vividness was created by including reflections and knowledge 
about the work and the organisation in general. The Falcon continued with their 
well-established and now SHEA-approved IQA work. However, considering the 
resources, time, and work invested in preparing and carrying out the pilot’s activities, 
issues about the actual value added from the review were raised. The HEI side was 
concerned that the pilot may have contributed to a growing sense of insecurity. The 
decisions did not include any information about what was actually missing in the 
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failed IQA systems, and thus it remained unclear what actually had to be 
accomplished to pass. 


Perspectives on the Pilot: Work, Actors, and Knowledge 


We would like to highlight some observations relating to the questions raised in the 
beginning of the chapter on enactments, actors, knowledge forms, and governing. 


Expansion of QA Work 


As one of our informants told us, an effect from the 2016 EQA system and the insti- 
tutional reviews may be that HEIs “start doing much more than what would really 
be required, just in order to be on the perceived safe side” (Falcon Quality 
Management Staff 1). This general expansion of practices and activities linked to 
QA is a discernible trend in the empirical data presented in this book. QA expansion 
can be seen as an important driver within a willing community (Jacobsson and 
Nordström 2010). This tendency to “do more QA” may be further energised by the 
pilot, as the informant above implied. Moreover, the comprehensive work within the 
SHEA and the external assessment panels to establish a systematic assessment 
framework proved to be very challenging and continued to elicit challenges for all 
parties. For example, authoritative decisions in terms of pass or fail grades triggered 
the HEI actors to analyse and compare decisions, but it was also accompanied by 
the insight that assessors’ work is — and cannot be anything else than — subjective. 
Thus, it varies between assessment panels and across time and space. 

When mapping the entire pilot process, it becomes evident that this kind of exer- 
cise implicates a huge amount of work and various forms of work that ultimately 
serve the purpose of governing. As noted by Clarke (2012), these efforts: 


[]ike all other forms of human labour, involve practices of transforming things. Whether 
the objective is to govern populations, projects, problems or processes, work (and people to 
do the work) is essential. Forms of work (and types of worker) are a condition of possibility: 
that populations might be regulated; that projects might materialize; that problems might be 
resolved or that processes might run smoothly. (Clarke 2012, p. 209) 


The remark about work as an engine of transformation might seem obvious, 
particularly in relation to EQA practices, because a basic rationale underlying these 
exercises is orchestrating qualitative changes or improvements. Even so, we think 
an analysis of actual work is productive for our purposes. For instance, our attempts 
to make work visible have displayed examples of when evaluative practices do not 
“work”. As noted by Clarke (2012, p. 209), “these forms of work are also the condi- 
tion of other possibilities, in which the anticipated or desired outcomes do not 
materialise”. 
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We would also like to highlight the amount of work and how the expectations 
around this work are framed and formed. On one hand, this capacious labour is 
required by the SHEA and manifested in, for instance, the assessors’ assignments 
and the HEIs’ facilitation of assessors’ data collection (i.e. to produce and deliver 
documents, to actively take part in interviews, to respond to decisions, etc.). On the 
other hand, there is no external limit on how much work HEIs can actually put into 
the process, other than the restrictions of deadlines at certain stages in the process. In 
our data, we observe that HEIs tended to expand their work and that certain forms of 
work were intensified within and between organisations. The HEIs engaged in vol- 
untary activities of knowledge exchange and in various forms of collaborations in 
national networks, such as the ASHEI. The pilot evidently required a lot of time and 
work in the studied HEIs, at least for top- and faculty-level management and for QA 
staff. When we asked about the amount of resources put into the pilot, however, the 
HEI actors hesitated. Despite the process’s careful planning and organisation in 
terms of activities and people, issues of time, money, and working hours were largely 
absent. In a sense, this “invisibilisation” of the resource aspect of QA work is to be 
expected as the actual work is embedded within the rationale of QA and enhance- 
ment. Such work is supposed to be integrated into all other forms of work within the 
organisation, and separating QA work from other work and activities is neither pos- 
sible nor desirable. As noted by Power (1994), the scale and the complexity of organ- 
isations were reasons for the institutionalisation of this supposedly cheaper mode of 
“evaluation of evaluation”. Power’s concern is that indirect forms of quality control 
have become rituals based on compliance without substance or relevance to organ- 
isations (Power 1994). Our findings also show that EQA is not entirely functional for 
the HEIs. Rather, EQA is to be understood as a state mode of governing HEIs. 


The Work of Translations 


Work is also carried out in the numerous translations of these processes. Our data 
show how HEIs struggle to make IQA systems fit the self-evaluation template; 
assessors struggle to understand the assessment criteria and the HEIs’ self- 
evaluations, and the SHEA staff struggle to reinforce valid and comparable 
judgements. Assessors and the assessed have to produce a fit between rules and 
cases. Even common concepts provoked uncertainty and translation processes. The 
assessment panels consisted of hybrid and potentially contradictory formations that 
collaborated under specific circumstances. They were also traversed by a particular 
“logic” inherent in the ESGs and the SHEA guidelines that produced indeterminacy 
and required them to resolve tensions, paradoxes, and contradictions. In other 
words, they had to translate from one form to another. The expectations were that 
such rules, the organisation and forms of practices within the pilot, and the personal 
qualities of assessors would enable politically desired equivalent and comparable 
review decisions (Standing Parliamentary Committee on Education 2016). From 
our interviews, we learn that this is a potentially impossible challenge. 
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Our focus on work also encourages us to consider those who perform the work. As 
noted by Clarke (2012), “[t]hese are not just ‘hands’ or “bureaucratic functionaries’ 
but need to be understood as having social characters, positions and dispositions 
that are formed in social relationships and trajectories” (Clarke 2012, p. 213). 
Throughout our project, we observed that persons working with EQA and its 
management and development are mobile and well networked. They move between 
HEIs, government agencies and associations, and various kinds of organisations. As 
we discussed earlier in this chapter, they tend to share knowledge and previous 
experiences. In this book, we use “qualocrats” to denote this somewhat heterogeneous 
group of actors who have taken on the authority and mission to, on behalf of the 
higher education sector, move between various domains to translate and promote 
certain forms of knowledge in and of QA. They carry highly valued expertise related 
to enacting and managing EQA, including the institutional review we studied in this 
chapter. Similar to Enders and Naidoo’s identification of “audit-market 
intermediaries”, we also observed “the institutional work of the new professionals 
who make sense of, buffer and translate institutional pressures” and how “normative 
frameworks and expectations [are] de-coupled, hybridised and sedimented” (Enders 
and Naidoo 2018, p. 10). Even if their approach is institutionally oriented, the 
attempt to “bring actors back in” to the institutional analysis is promising. Our 
project findings support such an argument, and we suggest that the qualocrats 
identified in our study are central actors for understanding important higher 
education transformations and their possible implications. 

The qualocrats also contribute to the formation of what has been labelled a “qual- 
ity assurance community”: 


A “quality assurance community” is thus both fabricated and projected (Grek et al. 2009), 
conceived as a homogenised collection of individual stakeholders “committed to continuous 
quality improvement” (ENQA 2010: 3). (Brady and Bates 2016, p. 70) 


Within this community, higher education is understood in particular ways. An 
orientation towards goals and results is prominent, which also implies a rational, 
quality-oriented, and constantly improving organisation (Brady and Bates 2016). In 
their work, qualocrats use and promote certain forms of knowledge, which we find 
important to highlight. We observed a certain vocabulary and particular expertise in 
QA. The knowledge base is largely derived from TQM. Historically, these ideas 
stem from management theories such as Taylorism, scientific management, and the 
human relation school and are imported from the industry and from business 
(Newton 2000, 2010; Palmer and Saunders 1992). 

Qualocrats are experts (Barrow 1999; Normand 2016), and their work can be 
understood as an activity of expertise: 


Expertise is a specific activity of knowledge production participating in a process of nego- 
tiation and orientation of public policy. This knowledge is technical and comes from profes- 
sionals working in administrations, international organisations, universities and other 
higher education institutions, agencies, think tanks or interest groups. (Normand 2016, 
p. 131) 
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Technical knowledge here must be understood broadly because expertise also 
involves know-how, such as the capacity to successfully engage in dialogue and in 
guidance (Normand 2016). While qualocrats draw on scientific legitimacy — where 
TQM offers general ideas — their expertise is often “based on experience and social 
fame acquired for years” (Normand 2016, p. 131). 

Such an understanding of expertise resonates with our interest in exploring vari- 
ous forms or phases of knowledge (Freeman and Sturdy 2014). Looking at the pilot, 
it is clear that certain key actors are particularly important, particularly qualocrats 
due to their embodied knowledge and expertise. They also possess the skills and 
networks to rapidly update this knowledge and to incorporate new information they 
learn from meetings with each other. Inscribed knowledge (e.g. in the ESG, assess- 
ment materials, decisions, etc.) is also crucial in the pilot’s work. Qualocrats draw 
on their embodied knowledge to interpret and translate such written material to 
make it “actionable” (Grundmann 2017, p. 31). As noted by Normand (2016, p. 156) 
“the worlds of expertise [and qualocrats] are plural and they have their own logic of 
action which, contrary to what the technocrats think, cannot be prescribed in 
advance through quality frameworks and protocols”. 

Importantly, however, inscribed knowledge “entails particular ways of seeing, 
thinking and knowing and serve to constrain and discipline interactions with the 
world and with one another” (Freeman and Sturdy 2014, p. 11). Our data illustrates 
how this is played out in the various stages of the pilot process. In accordance with 
our focus on governing as a verb, we take a particular interest in what actors do with 
their knowledge. As noted by Freeman and Sturdy (2014), enactment may give rise 
to new knowledge beyond what was previously embodied or inscribed. In this 
context, meetings are crucial for such enacted knowledge. In this pilot, meetings 
were indeed “the basic unit of the policy process” (Freeman and Sturdy 2014, p. 12). 
Qualocrats and their different networking and meeting activities resemble what 
Haas (1992) described as an epistemic community, i.e. “a network of professionals 
with recognised expertise and competence in a particular domain and an authoritative 
claim to policy-relevant knowledge within that domain or issue-area” (Haas 1992, 
p. 3, cited in Freeman 2008). As Freeman (2008) points out, such epistemic 
communities have an important role in processing uncertainty. In the pilot, there 
was a continuous need for the: 


[i]nterpretation of inadequate or complicated and sometimes contradictory information, 
and on a corollary requirement to stabilise and sustain the flow of information and 
interpretation to policy makers. Understandings of complex systems, in turn, have come to 
cast the process of communication as generative, and systems as in some sense creating 
both themselves and their environments. (Freeman 2008, p. 3) 


Meetings, says Freeman, are essentially concerned with “the definition of the 
problem they are designed to address” (Freeman 2008, p. 17). In other words, meet- 
ings are sites of enactment and learning, and they are performative in the sense that 
objects of mutual interest are constructed through discussions. 


178 J. Lindgren et al. 
Governing by Piloting 


This leads us to a final observation on the pilot’s governing dimension, what we 
have labelled “governing by piloting”. The pilot’s very form and deliberately 
temporal design include work that links people, places, policies, practices, and 
power in particular ways (Clarke 2015) and in the form of a particular “governing 
project”. Framing it as a pilot opened up spaces for mutual adjustments, learning, 
and dialogue and a priori reduced the “stakes”. In these processes, there are still 
certain anticipatory governing signals sent to HEIs in the pilot and to HEIs not in the 
pilot. In this case, one governing signal was “to be on their toes”. It turned out that 
most HEIs involved in the pilot got a failing grade, which clearly sent a message that 
“this system can bite, too”. At the same time, there is a continued emphasis on soft 
norms in terms of dialogue, trust, mutuality, enhancement, and openness. This 
orientation and “affective atmosphere” were prominent in the process leading up to 
the 2016 EQA system and in the pilot. In sum, we suggest that this orientation 
conveys somewhat contradictory signals to actors that will be consolidated, 
processed, and “done” in further enactments within and beyond the regular and full- 
scale implementation. 


Finally 


This chapter analysed the pilot of the 2016 EQA institutional reviews. The chapter 
concentrated on the work of different actors involved in these processes and 
structured the “story” via three main chronological stages in the pilot, i.e. 
preparations, data collection, decision-making, and finally post-decision work, 
which included some partial adjustments of the institutional reviews. We empirically 
emphasised the amount of work and the forms of work done by various actors in 
these enactment processes. In this context, we also noted the numerous translations 
that continuously take place — even if the planning of the process and the level of 
detail in the prepared guidelines are quite thorough. One important group of actors 
that are enacting and brokering EQA knowledge and the work and activities that go 
along with it are the so-called qualocrats. We identified them as particularly central 
and important actors, and we will continue to highlight this group, among other 
things, in the upcoming concluding chapter. 
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Abstract In this chapter, we revisit our observations on Swedish higher education 
from the previous chapters. We initially discuss the relation between governing, 
evaluation, and knowledge. Then, we go back to the ideal typical notion of 
increasingly institutionalised evaluation machinery and locate the important work 
of what we label as qualocrats and what may be termed the burden of judgements 
within this overall frame. We finally discuss the expansion and the increasing 
complexity of evaluation and quality assurance work in higher education and point 
to some possible implications and problems. 


The Governing—Evaluation—Knowledge Nexus: How 
Evaluation Makes Knowledge Work for Governing 


In this final chapter, we highlight some of our observations on evaluation and qual- 
ity assurance (EQA) in Swedish higher education. Our explorations have been 
guided by an interest in how governing, knowledge, and evaluation are bound 
together, as denoted by the term “nexus”. In this work, we had a particular interest 
in studying how the features of evaluation as a social practice make knowledge 
work for governing, involving different actors, levels, and perspectives nationally 
and internationally. The broader context of the book is concerned with new forms of 
epistemic governance of higher education. Our starting point was “that the 
production of knowledge does not just belong to scientists: it is distributed among 
heterogeneous experts with a central position to give advice and to guide 
policymakers” (Normand 2016, p. 129). We acknowledge the role of knowledge as 
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an instrument of power and legitimisation involving language, norms, decisions, 
beliefs, aspirations, negotiations, and actions that emerge from various actors and 
institutions (Normand 2016). Throughout the book, we tried to “push theorizing to 
a thicker description of how actors are socially embedded and how they employ 
(even if implicitly) that position in influencing others” (Alasuutari and Qadir 2014, 
p. 71) on the basis of particular conceptions, understandings, and forms of 
knowledge. Our ambition here is to pinpoint the particularity of evaluation in higher 
education as a social practice that makes knowledge work for governing. In the 
book, we provided examples of how evaluation produces “constitutive effects” 
through forms of knowledge (Dahler-Larsen 2015, p. 24). In the words of Dahler- 
Larsen (2015, p. 24), “the constitutive effects of evaluation extend to how we know, 
to our sources of knowledge” as “people change their interpretations and their 
actions as a result of... knowledge (officially regarded as knowledge or not) which 
are touched upon or enrolled by evaluation regimes”. The speciality of evaluation in 
higher education is that it includes work on preparing, making, receiving, and acting 
on judgements of the quality of intangible phenomena, such as processes, 
responsibilities, routines, competences, relations, and support. Thereby, evaluation 
constitutes a certain form of epistemic governance that presupposes, uses, and 
produces certain forms of knowledge about a present condition that is extrapolated 
into a desired future. In this sense, the governing—knowledge-—evaluation nexus is 
bound together by a utopian dimension in which evaluation is used to generate, 
promote, and mediate certain forms of knowledge to pledge (constant) change in 
ways that fit well with governing ambitions. 

In the first chapters, we initially outlined the historical development of national 
EQA in Swedish higher education and situated this intensified political striving 
within an international and primarily European context. The politics of (interna- 
tional) comparison and the perceived political need to govern the design and the 
expansion of EQA, as well as its being taken for granted as a part of higher educa- 
tion governing, were features identified and emphasised in the initial chapters that 
illustrate the complexities of the governing, knowledge, and evaluation interac- 
tions. We also illustrated that the EQA policy’s activities, processes, and the pace 
by which they are produced are important when unpacking subsequent enactments 
and governing work. 

We documented the manifestation of these evaluative activities in terms of evolv- 
ing “evaluation machines” (Dahler-Larsen 2012). In this context, we pointed to the 
important role of the work of particular actors, such as vice chancellors, interest 
groups and organisations, the media, policy professionals, intermediaries, and “qua- 
locrats” (see below), who, via evaluation, become situated as enforcers, enablers, 
and governors. As our account became more empirically fine-grained towards the 
end of the book, these overall findings were analysed in more detail and with an 
explicit intention to be empirically exhaustive. In these analyses, we have shown 
how internal and external evaluation and quality assurance policies and practices are 
created, merged, and enacted (i.e. “worked’’), and we pointed to the numerous activ- 
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ities and the amount of labour that go into these processes. We also discussed how 
the forms and qualities of “knowledge” are promoted or advanced by this work and 
how knowledge transforms between phases through various kinds of action 
(Freeman and Sturdy 2015). Doing EQA thus entails various forms of translations 
and results in a continuous expansion of evaluative work and activities. We label this 
emerging process a evaluation machinery. 

This final chapter is divided into two main sections. First, we continue the above 
discussion, initially by revisiting the ideal typical notion of an increasingly 
institutionalised evaluation machinery and by looking into how its engineers and 
main operators, the qualocrats, can be understood. We also discuss the art of making 
judgements, a central aspect of “binding together” forms of knowledge to make 
evaluation work. In the second section, we discuss the expansion in terms of the 
increasing complexity of EQA work in higher education and recognise some 
possible implications and problems in terms of resources and sustainability. 

One important argument in the first section is that there are different ways in 
which evaluation makes knowledge work for governing. The first way corresponds 
with “conventional” interpretations of epistemic governance, or soft governance. 
Here, power and knowledge operate through certain automated technologies that 
trigger and stimulate behaviours. The way comparisons are introduced in the 
evaluation machinery offers one such example. Comparisons — and the very notion 
of comparability that comparisons between HEIs and/or programmes by the state 
and/or by students are possible — automatically initiate new behavioural protocols 
based on informed work within HEIs. The second way evaluation makes knowledge 
work for governing is located in the concrete and often meticulous daily work and 
interactions of actors to make evaluation work. As demonstrated in previous 
chapters, such work involves different forms or phases of knowledge, like inscribed 
knowledge. It also involves work to produce or act based on texts, such as policies, 
evaluation criteria, or self-evaluations. The work may also implicate meetings 
where language and embodied forms of knowledge are enacted to process, transmit, 
or produce new knowledge. This work includes numerous translations and 
mediations that may cause misunderstandings, inconsistencies, or problems 
traditionally described in terms of validity and reliability. On a somewhat different 
level, these laborious activities may add little to promote organisational improvement. 
After a demanding external evaluation, actors within HEIs may conclude that they 
did not learn anything about “the quality” of their work or their organisation that 
they did not already know. Alternatively, evaluands may find the evaluation report 
inaccurate, obscure, or otherwise difficult to use for purposes of organisational 
change. Thus, programmatic (normative) and technological (operational) elements 
are not always perfectly aligned (Power 1997), implying that situations occur when 
evaluations do not work according to official transcripts or ideals. Our point here is 
that, even under such conditions, evaluative activities still draw actors’ attention to 
policies and consolidate actions related to policy in particular ways. Hence, 
regardless of whether evaluation actually “works”, the processes initiated by 
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evaluation can still make knowledge work for governing, albeit not exactly how the 
“evaluation machine engineers” may have intended. 


The Emerging Evaluation Machinery 


The previous chapters have documented the rise of Swedish EQA in higher educa- 
tion. Up until rather recently in the history of higher education, anything of its kind 
simply did not exist. In fact, just a few decades ago, any intrusion by external agen- 
cies on HEIs, their inner work, and freedom, was unthinkable. As a reminder of the 
pre-EQA zeitgeist, the Robbins Report on higher education, an important post-war 
policy document in the UK, offers a good example. In this report, it was emphasised 
that: 


such freedom is a necessary condition of the highest efficiency and the proper progress of 
academic institutions, and that encroachments upon their liberty, in the supposed interests 
of greater efficiency, would in fact diminish their efficiency and stultify their development. 
(Committee on Higher Education 1963, p. 228) 


Today, on the other hand, the national evaluation machinery is a permanent, institu- 
tionalised feature in HEIs, with implications for language, culture, ideas, practices, 
and artefacts produced by these institutions. The permanence of this machinery and 
the almost naturalised status of its basic and underlining principles has evolved over 
time in accordance with the rise of what has been described as the making of the 
evaluation society (Dahler-Larsen 2012), the audit society (Power 1997), the evalu- 
ative state (Neave 1998), and audit cultures (Strathern 2000). The general process by 
which such permanence is socially established has been the focus of much social 
theory. As noted by Bourdieu (1977, p. 164), “[e]very established order tends to 
produce (to very different degrees and with very different means) the naturalisation 
of its own arbitrariness”. In this particular case, belief — often taken for granted — in 
ideas about fundamentally uncertain and fragile concepts, such as “quality” and 
“continuous improvement”, is particularly fascinating. Travers (2007) claims that 
quality assurance (QA) is simultaneously pervasive and diffuse. This means, for 
example, that any consumer of EQA discourse will struggle to work out the actual 
meaning of the concept of quality. In the words of Warzecha (2017, p. 11), “the 
extensive explanations and notes that often accompany this term support the impres- 
sion that something is defined that in fact remains completely in the dark”. At the 
same time, quality has established itself as “the cross-boundary norm against which 
all areas of HEIs ought to strive against” (Schoug 2006, p. 65). One of many pos- 
sible explanations of the pervasive naturalism of “qualispeak” is that it is fused with 
academic values, such as transparency and accountability (Lorenz 2012, p. 625, see 
Shore 2008). As stated by Shore (2008, p. 291): 


This may also be part of the reason why audit culture is so difficult to contest; the university 
environment has become so steeped in managerial principles and practices that it is difficult 
to find that Archimedean point outside of the system that enables us to critique it. 
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In this book, we have demonstrated the role played by various technologies within 
the evaluation machinery. In this context, technologies are conceptualised not as 
material aspects, such as communication technology or other tools, but as methods 
and procedures for governing human beings that operate in a Foucauldian sense. We 
discern how technologies, such as visibility, comparability, standardisation, 
economic incentives, and rewards and sanctions, have become embedded in the 
national machinery over time. We can also discern that the techniques themselves 
have mutated over time. 

The technology of visibility is one example of how such change is introduced for 
knowledge to work for governing. Visibility offers means to reduce complexity 
through abstraction, categorisation, and data concentration (Dahler-Larsen 2012). 
For instance, visibility was originally an effect of state attempts to “scan” (Gröjer 
2004, p. 64) or make all universities and their inner workings visible by means of 
producing comprehensive knowledge about the sector. This line of reasoning is 
congruent with Scott’s (1998) ideas on state vision and large-scale state measures to 
improve society and its institutions by simplifying local practices and making them 
legible during the twentieth century. Simplification and legibility are associated 
with “a synoptic view of a selective reality” that imposes a certain “logic on the very 
reality that [is] observed” (Scott 1998, pp. 11-14). 

Over time, visibility has transformed from a kind of “singular technology” oper- 
ating through the state centric gaze upon individual institutions into a “polycentric 
technology” of permanent visibility. The state agency has continued its observa- 
tions, but higher education institutions and actors have made themselves more and 
more visible not only to the state but to themselves and other institutions, prospec- 
tive students, stakeholders, and the general public. As noted by Foucault (1977) in 
his studies on the concept of panopticon, visibility is an anonymous power that 
produces efficiency, responsibility, and discipline. The potential of visibility in this 
respect has been part of management thinking for a long time. As evidenced by the 
Hawthorne experiments in 1924, the observer’s effect on behaviour was used to 
increase industrial productivity. Visibility as a technology is thus based on general 
knowledge, as perfected by Mayo (1946) and proponents of the Human Relations 
School (the forerunner to total quality management, TQM), namely, that people 
behave “better” when under observation compared to when left to operate on their 
own. Interestingly for us, visibility within the evaluation machinery has also been 
refined along these lines, as it has become a part of monitoring and follow-up 
systems that are increasingly taken for granted. For instance, recent developments 
illustrated in the previous chapters, in which EQA has incorporated enhancement 
ideas emphasising the value of anchoring, dialogue, and involvement, can indeed be 
discussed in relation to lessons drawn from Mayo (1946). Mayo linked governing 
style and morale levels to productivity levels, pointing to the need to see and listen 
to employees and show interest in their working conditions to increase motivation, 
even if conditions did not change. Such a case history complements our understanding 
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of the rationale behind attempts to include all aspects and divisions of HEI 
organisations in continuous, expanding attempts to improve quality. 


The Importance of Human Interaction 


Indeed, technologies are examples of the “automated character” (Dahler-Larsen 
2012, p. 180) of the evaluation machinery. Even so, we have also shown how 
embodiment is crucial for the construction and operation of the machinery. In this 
sense, our observations could be used to reconsider the line of reasoning provided 
by Dahler-Larsen (2012, p. 180), who emphasises how formalisation reduces “the 
need for expertise, wisdom, and educated staff’. Our data has shown that 
formalisation in the context of Swedish national EQA in the form of the detailed 
frameworks and guidelines intended to increase homogeneity and comparability 
requires substantial human interpretation and translation. In this context, it is 
interesting to recall Wittgenstein’s reminder that following rules is a social practice 
and that we need tacit forms of knowledge even to follow the simplest rules: 
“following a rule is [in fact] not like the operations of a machine” (Wittgenstein 
quoted in Taylor 1995, p. 168). According to Wittgenstein, rules are followed on the 
basis of an understanding that “is always against a background of what is taken for 
granted, just relied on” (Taylor 1995, p. 167). The operation of an evaluation 
machine demands such forms of knowledge. Then again, the scripted nature of the 
machine and the demands it places on representation distinguish it from aspects of 
more everyday social practices where rules are primarily followed on the basis of 
automated social understanding or “habitus” (Bourdieu 1977). Our data show that 
frameworks and guidelines do not contain the principles of their own applications. 
In our interviews, we have asked our informants how they perceive certain common 
concepts that are frequently used in the context of the evaluation machinery (such 
as “ensure” [säkerställa] or “quality”). We have recurrently observed how our 
informants become uncertain when asked these questions. In this sense, policy 
demands for homogeneity and comparability, which are based on these ambiguous 
concepts, seem to feed insecurity and the need for collective labour. Face-to-face 
meetings and negotiations between actors are often necessary for the machines to 
work. When actors do not trust their own interpretations, they must meet with one 
another to help each other “learn” to understand rules and “make sure” that they 
have achieved a mutual understanding of the rules. In this sense, physical meetings 
are crucial arenas for such interaction and work (Freeman 2008). 

The importance of human interaction is a central finding of our project. As out- 
lined above, technologies would — at least in theory — result in evaluation machines 
distinguished by an automated character with reduced need for human intervention 
and expertise. Such a version of evaluation machines would resolve problems 
related to coordinating dispersed knowledge following Hayekian ideals (Hayek 
1945; see the chapter “Hayek and the Red Tape: The Politics of Evaluation and 
Quality Assurance Reform — From Shortcut Governing to Policy Rerouting”) and 
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ultimately result in an anonymous, self-governing disciplinary power working in 
accordance with ideas presented by Foucault (1977). A possible way to understand 
why this has not been the case is to consider the difference between complex and 
complicated systems and the implications of this difference on higher education 
governance and evaluation. According to Glouberman and Zimmerman’s (2002) 
distinction, complicated problems (like sending a rocket to the moon) are intrinsically 
different from complex problems (like raising a child or managing a health system). 
Whereas complicated systems are planned, predictable, and engineered, complex 
systems are unique, relational, and uncertain because they consist of multiple 
interconnected components or actors that change adaptively (Glouberman and 
Zimmerman 2002). This rough ideal-typical distinction is informative because it 
draws attention to how actors and their forms of communication are imperative for 
knowledge to work for governing in the realm of the evaluation machinery. 

We have noted that the evaluation machinery of Swedish higher education is 
simultaneously a complicated system and a complex system. As a complex system, 
it draws on technologies that make things happen seemingly without premeditated 
orchestration. However, it is also a complicated system in the sense that the scale of 
system design and the requirements for coordination and interpretation of 
components by specialised experts are significant. This means that the more concrete 
acts of making things visible or performing comparisons are far from spontaneous 
or straightforward. As argued by Mowles (2014, p. 163), the above distinction 
between complicated and complex systems fails to acknowledge that problems of 
the former kind — presumably instrumental and straightforward — always comprise 
“widespread mutual adaptation and improvisation, disagreements, lacunae, the 
unexpected and the contingent”. Internal and external forms of evaluation involve 
taking processes of management and teaching apart and analysing the details. It is a 
highly engineered and detailed system that requires evidence, planning, management, 
role definitions, and alignment. It also requires labour-intensive human interactions 
in the form of meetings whose purpose is to identify and eliminate fallacies caused 
by the differentiation and fragmentation built into the machinery. Process elements 
in EQA may be forgotten, described incorrectly, evaluated incorrectly, put in the 
wrong order, or separated or joined in the wrong places. Unnecessary process 
elements may be added, or mutual dependencies of process elements may be 
incorrectly described (Warzecha 2017). This work is inevitably done by actors, and 
in our project, we have identified particular forms of work in which actors — namely, 
the qualocrats — are situated as both engineers and operators of the evaluation 
machinery. Next, we turn to these actors and their background, work, and knowledge. 


The Work of Qualocrats 


We envision qualocrats as powerful actors in a form of government or rule (from 
Greek kratia and kratos) in the name of quality and quality improvement. This group 
of actors embraces many traditional bureaucratic values and ethics, but its power is 
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concentrated not in administrative bureaus but in webs of organisations, institutions, 
policies, practices, and work related to QA and evaluation. Along these lines, 
Travers (2007, p. 11) has discussed the rapid development of QA as a new “specialist 
occupation” in society. Travers (2007) traces the origins of the quality movement to 
twentieth-century ideas about TQM in Japanese and American industry. These ideas 
eventually made their ways into the public sector as a form of “reinvention of 
government” (Travers 2007, p. 15). Qualocrats belong to this wider tradition, but no 
mission statement in terms of introducing management ideas from the private 
industry to the public sector guides them. Overall, qualocrats do not constitute a 
homogenous group, and they come from somewhat different occupational 
backgrounds. We thus want to emphasise the fluidity that marks this community of 
actors. A small fraction may work full-time with quality issues as quality managers 
in HEIs or as vice chancellors, whereas others work part-time. Directors of studies 
or teachers within HEIs may be “interpelled” as qualocrats and carry out qualocrat 
work (e.g. prepare and assemble information or answer questions) during specific 
evaluations. Some are asked — and agree — to work as external reviewers, and others 
function as quality experts that are consulted by HEIs. Notably, most qualocrats are 
trained researchers, and some manage to pursue their scholarly careers parallel to 
their work as qualocrats. 

Some qualocrats in our study have a short career path within management and 
have specialised in issues of QA through national and international networks and 
organisations. Through deliberative attempts during recent decades, to encourage 
involvement of students in governance structures as stakeholders, such as the 
Bologna Process, QA issues now offer a career path for students within student 
unions and the European Association for Quality Assurance in Higher Education’s 
(ENQA’s) member agencies, both nationally and internationally. Other qualocrats 
have long backgrounds within the HEI sector and the evaluation field. This group 
can use their experience to bridge “old” knowledge with evolving evaluative ideas 
and systems in HEIs that must adhere to new societal and economic demands. Such 
bridging is enabled by contemporary outlooks on QA rooted in ideas that have long 
traditions outside the HE sector. 

One way to initially locate the stature, knowledge, and work of qualocrats is to 
situate this group of actors within wider “professional landscape” (Brante 2013). 
Although their work is strongly linked to TQM and the Swedish subject 
Kvalitetsteknik [Quality technology, Quality management, or QA], their educa- 
tional background is diverse, and there is no common doctoral programme anchor- 
ing them within the HE sector. Qualocrats are interdisciplinary, which means that 
“there is no robust, systematic, generally recognised, shared paradigm that unites 
practices” (Brante 2013, p. 6). Hence, under the surface of the seemingly coherent 
public manifestations of qualocrats, there may be competition and struggles over 
“jurisdiction and the basic doxa” (Brante 2013, p. 6). Just like semi-professionals, 
such as social workers and teachers, qualocrats have expanded in sync with the new 
layers of bureaucracy. To a great extent, they are governed by regulations and pol- 
icy — but on the other hand, they have also established themselves as policy actors 
who influence both the making and enactment of EQA policy. As a pre-professional 
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group, qualocrats have managed to create a niche for themselves in the wake of 
transformations within HEIs as “new modes of rationality in the public sector (such 
as New Public Management) has generated demands for various types of leadership 
experts, management consultants, specialist consultants” (Brante 2013, p. 8). 

The identification of new groups of actors in the emerging field, such as EQA 
experts, consultants, officers, coordinators, vice chancellors, and vice deans, has 
been noted elsewhere. One strand of research classifies these actors as members of 
the growing management within HEIs who have been profoundly transformed by 
New Public Management and marketisation (Hall 2012; Lorenz 2012; Alvesson and 
Spicer 2016; Graeber 2018a, b). In pejorative terms, Graeber (2018b, not paginated) 
writes about so-called taskmasters who create extra work for academic staff who 
find themselves “spending less and less time studying, teaching, and writing about 
things, and more and more time measuring, assessing, discussing, and quantifying 
the way in which they study, teach, and write about things”. 

This criticism aside, most qualocrats in our study are defenders of general 
Humboldtian ideals in terms of academic freedom, and they see themselves as 
protectors of collegial norms and autonomy. Nevertheless, the qualocrats’ position 
is far from easy, and they can be seen as relays between two narratives (Jarvis 2014, 
see also the chapter “Hayek and the Red Tape: The Politics of Evaluation and 
Quality Assurance Reform — From Shortcut Governing to Policy Rerouting”): On 
the one hand, they advocate traditional academic ideals and virtues, but on the other 
hand, they work to establish “regulatory regimes that seek to manage, steer and 
control the sector in ways that serve the interests of the state and the economy by 
applying specific ideational motifs about efficiency, value, performance, and thus 
the economic worth of the university to the economy” (Jarvis 2014, p. 156). 

As we have indicated in the book (see the chapter “National Evaluation 
Systems”), the previous history of the national evaluation machinery shows that 
formation of qualocrats as a distinct group could have taken an alternative route. In 
the 1970s, early prototypes of the evaluation machines were engineered by scholars 
from the educational sciences who set up prospects for the future in terms of system 
evaluations that included information of “forms of teaching, teaching methods, 
students’ pre-conditions, study environment, study habits, teachers’ working 
conditions” (Gréjer 2004, p. 61). As it turned out, evaluation machines transmuted 
over time and made such pedagogical queries obsolete: There was no official interest 
in these aspects of social life in HEIs. In addition, the machine operators morphed 
into the current heterogeneous body of actors. 

What, then, do qualocrats do? Shore and Wright (2000) identified such experts 
within the new political technologies in higher education almost 20 years ago and 
argued that they fulfil four main roles: 


First, they develop a new expert knowledge and a discourse which create the classifications 
for a new framework or template of norms, a normative grid for the measurement and 
regulation of individual and organizational performance. Second, their grid and expertise 
are used for the design of institutional procedures for setting targets and assessing 
achievements. Third, certain of these experts staff and manage the new regulatory 
mechanisms and systems, and judge levels of compliance or deviance. Fourth, they have a 
therapeutic and redeeming role: they tutor individuals in the art of self-improvement and 
steer them towards desired norms. (Shore and Wright 2000, p. 62) 
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Qualocrats, as we have come to know them in our project, correspond well with this 
characterisation, but there are some additional comments to be made. First, we 
acknowledge the dual function of qualocrats as engineers and operators of the 
evaluation machinery. We have noted in the book how boundaries between evaluators 
and those subjected to evaluation have been gradually dissolved by new modes of 
deliberation and influence over forms and content in national EQA. As a 
consequence, qualocrats — who play a central role in such processes — acquire 
knowledge that can be “brought home” and used in local organisations (for instance, 
in HEIs). Such knowledge may be manifested in strategies to organise responses to 
specific EQA. One example in the data, mentioned in the chapter “Re-launching 
National Evaluation and Quality Assurance: Governing by Piloting”, was the 
establishment of a “liaison central” during site visits that served to coordinate actors 
who took part in interviews, their responses, and their emotional reactions. Such 
modes of qualocrat-led professionalisation of internal strategies to deal with external 
evaluation can determine the formal outcome of the entire exercise — it can be the 
difference between a pass or fail grade. In addition, the role of qualocrats is also that 
of mediators and translators, as their work has become increasingly regulated by 
inscription in frameworks and guidelines. We also see a development regarding the 
fourth role, steering individuals towards “desired roles” (see above). Particularly in 
the period after the much debated 2011-2014 national EQA system, qualocrats 
functioned as experts in facilitating “compensatory legitimation” and dealt with 
problems of distrust on the national and local level among actors in HEIs in attempts 
to “socialise people into certain attitudes and dispositions towards authority, 
performance, cooperation” (Weiler 1983, p. 273; see also Neave 1998, pp. 270- 
271). Finally, as we have argued above, the expertise of qualocrats and the discretion 
they exercise embody sets of expectations and beliefs that tend to be dissimulated as 
a matter of “technical” procedures or ideologically and theoretically innocent 
strivings for quality. In the next section, we explore this issue further by turning to 
the centre of evaluative work — judgement making. 


The Burden of Judgement 


The previous accounts of EQA in this book have recurrently touched upon the 
judgement making work. As we pointed out in the first chapter, judgement is an 
inherent feature of evaluation in which certain knowledge becomes activated and 
thereby used for governing. Parts of this judgement work are done by qualocrats, 
and some external assessors could very well be labelled qualocrats. However, all 
external assessors are “charged with making decisions subject to standards set by a 
particular authority” (Dworkin 1978, pp. 31-32 in Molander 2011, p. 321). In our 
study, the “authority” is the state, as represented by the Swedish Higher Education 
Authority (SHEA), and the “standards” are materialised in the inscribed knowledge 
laid down in, for example, the designs of the national EQA systems and the 
instructions and guidelines for the training of assessors. These standards, or 
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conditional frames, constitute an inscribed “discretionary space” (Wallander and 
Molander 2014, p. 2) for the external assessors. 

One of the conditions that frame assessors’ work, their discretionary space within 
the machinery, was the invention of a cut-off score in the 2001-2007 EQA system 
(see the chapter “National Evaluation Systems”). Linked to rewards and sanctions, 
cut-off scores condition assessors’ judgement making. The implications of cut-off 
scores are known from debates related to high-stakes testing, and in congruence 
with these discussions, we have seen how cut-off scores have been accompanied by 
sanctions, penalties, funding reductions, and negative publicity in the higher 
education sector. In addition, cut-off scores have resulted in heated discussions if 
they are perceived to be incomplete, flawed, or unfair because the consequences are 
based on judgements made by a relatively small number of assessors. Thus, 
judgement making involving cut-off scores tends to pull those subjected to evaluation 
and other audiences into this social practice in dramatic ways. Cut-off scores make 
it possible to judge HEIs’ internal QA systems as “failed”, meaning that they must 
take measures and be reassessed and hopefully judged in a second round as attaining 
the minimum standard for approval. 

Another interrelated condition is the detailed manuals produced for the 2016 
national EQA system, which were aimed at standardising judgements. In this work, 
the SHEA strove to direct external assessors’ judgements in particular ways by 
specifying a large number of perspectives, aspects, and indicators. This SHEA work 
was done to narrow the discretionary space for judgements across different 
assessment panels. In addition, in this EQA system, all perspectives, aspects, and 
indicators must be assessed as good enough for the HEIs to be judged with a pass 
grade. The external assessors are not allowed to let a strong result for one indicator 
compensate for a weak assessment result on another indicator. Only if all conditions 
are found to be satisfactory should the assessors assign the HEI or educational 
programme in question a pass. Finally, the 2016 EQA design prescribes that the 
external assessors must reach a unified judgement. They are not allowed to attach 
notes saying the decision was not unanimous. 

This way of defining external assessors’ discretional space expresses a means— 
ends relationship or an instrumental norm (Molander 2011; see also Wallander and 
Molander 2014). In the present case, this discretionary space thereby becomes 
reduced, in line with the SHEA’s ambition to enhance comparability across panels 
and HEIs, a process framed as ensuring equivalent and fair judgements. In spite of 
these ambitions, the external assessors had to rely on “discretionary reasoning” 
(Wallander and Molander 2014, p. 2) to reach a final, unified judgement. They had 
to engage in deliberations about the circumstances and traits of an individual case to 
come to a decision (Molander 2011, p. 330). We find that discretionary reasoning is 
an appropriate expression for characterising the work of the external assessors’ use 
of the various indicators set up by the SHEA to come to a decision on how to grade 
individual HEIs. Leaning on Rawls (1993), Molander (2011) discusses the premises 
of discretion and discretionary reasoning and points to the many difficulties that 
such reasoning often entails: Even if professionals “reason as carefully and 
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conscientiously as possible they may arrive at different conclusions” (Molander 
2011, p. 330). In this work, the external assessors’ embodied knowledge is enacted 
in the form of previous experiences from similar work, something valued by the 
SHEA in their recruitment of assessors. Despite explicit formulas of inscribed 
knowledge in guidelines, etc., informed judgements still require discretionary 
reasoning, and such reasoning may end up in “sensible disagreement” (ibid.). 
Borrowing Rawls’ expression “the burden of judgement” (Rawls 1993, Lecture 2, 
§2), Molander argues that the burden of discretion implies that consensus cannot be 
expected in certain areas by sensible persons (Molander 2011, p. 330). Relevant 
facts can be complex and contradictory, and there may be disagreements about how 
to weight different considerations, and interpretations can vary (Rawls in Molander 
2011, pp. 330-331). As noted in the chapter “Re-launching National Evaluation and 
Quality Assurance: Governing by Piloting”, not all external assessors were entirely 
comfortable with the SHEA’s demands that all panels are to agree on a unified 
judgement and not having the possibility of, for example, a strong outcome in one 
area compensating for a weaker outcome in another. Long deliberations were 
therefore occasionally needed to reach a unified agreement that all panel members 
could agree on. 

As mentioned above, a new condition of the 2016 national EQA system was the 
inclusion of detailed instructions to the assessors with a differentiation of a large 
number of perspectives, aspects, and indicators to be applied consistently in each 
assessment. Assessors had not been instructed in such detail in earlier national EQA 
systems. We may say that the SHEA wished to convey specific inscribed knowledge 
to its assessors. This knowledge became enacted as the SHEA staff organised 
training sessions with the assessors. The modes of inscribing knowledge in this 
highly complicated infrastructure of rules thus touch on the classic and recurring 
question through the history of science of whether a set or whole can be determined 
by its elements or parts. In this context, Warzecha (2017, p. 40), referring to Bertrand 
Russell and mathematics, has argued that “[flor process orientation — that is the 
dissection of all work processes — the following holds: the more differentiated the 
requirements are, the more probable it is that the number of wrong interpretations, 
of miscalculations and misunderstandings increases”. The burden of judgement is 
also a burden of differentiation — or as the popular saying goes: “the devil is in the 
details”. 

A different aspect of the burden of judgement is related to the dual purpose of the 
system to develop and control quality in higher education, as laid out in the system 
directions from the Swedish government (Government Petition 2015/2016:76). 
Evaluation and assessment theorists have long argued for the potential of assessment 
for development and learning, specifying formative assessments as assessments 
aimed at enhancing the quality of the characteristic at hand and summative 
assessment as the practice of making a final judgement on which to base decisions 
(e.g. Scriven 1967; Sadler 1989; Shepard 2000; Black and Wiliam 2009). In essence, 
the 2016 national EQA system is intended to combine formative assessment with a 
summative assessment in its control function. As noted above, the system includes 
a cut-off score and, therefore, a risk of being assessed as a failed HEI, which 
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suggests that the outcome of the judgements involves high stakes. According to our 
informants, this particular feature of the system design makes the control function 
more prominent and the development function less so. Instead of “opening up” to 
assessors, HEIs have incentives to portray themselves in a more favourable light or 
even try to hide weaknesses. 

In summary, the SHEA’s work of designing the framework for assessment, as 
well as the work of assessors in judging the quality of the HEIs’ internal quality 
assurance (IQA) systems, is laborious and difficult. The SHEA has invested vast 
energy and resources in dialogue with the HEIs to persuade them to embrace the 
2016 national EQA system in a positive way. This work also includes training and 
dialogue with the assessors with the aim of producing equivalent and fair assessments 
as part of the audit process. The external assessors were selected by the SHEA on 
the grounds of their embodied knowledge concerning academic work and EQA, but 
their discretionary reasoning was constrained by extensive manuals and frameworks, 
limiting the discretionary space. This tells us that the burden of judgement also 
relates to the epistemological assumption of rationality and automatization inherent 
in the idea of evaluation machines (Dahler-Larsen 2012). However, some evaluation 
theorists, such as Eisner (1999), stress a variety of opinions in judgements to support 
improvement and meaningfulness: “A life driven by the pursuit of meaning is 
enriched when the meanings sought and secured are multiple” (Eisner 1999, p. 658). 
Then again, such an evaluative design is not compatible with purposes of authoritative 
accountability. 


Expansion and Increasing Complexity 


In this final section of the book, we shift focus and theoretical gears to discuss the 
more general issues of expansion, resources, and sustainability. Our report on the 
proliferation of the national evaluation machinery in higher education prompts us to 
ask questions about costs that are relevant today and that may become even more so 
in the future as, and if, the expansion continues. As Dahler-Larsen (2012) has 
argued: 

The evolutionary imaginary inherent in the evaluation machines tends to increase the costs 


through steady expansion of the machines, but it also tends to keep the costs out of sight and 
discussion. (Dahler-Larsen 2012, p. 187) 


Even if many qualocrats and HEI actors in our study are concerned about this issue, 
for example, the increasing workload and personnel time that is subsumed in 
additional EQA activities, they also take part in designing, constructing, developing, 
integrating, engineering, operating, and feeding the machines. 

Posing questions about costs, resources, and sustainability is also a way to reduce 
forms of ontological reductionism and myopic vision. Evolutionary theory offers a 
tool to inform such choices by providing a historical perspective on social change. 
In this final section, we therefore turn to the American anthropologist Tainter (1988), 
whose work belongs to the broad tradition of Marx, Tonnies, Durkheim, and 
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Parsons. We do this to discuss investments in new layers and practices of qualocratic 
infrastructure that have been introduced as a state response to essential problems but 
which may have already reached or will eventually reach a point of diminishing 
marginal returns. 

In recent years, the literature on governance and complexity has become exten- 
sive. If we consider this book’s account of the relatively short history of EQA in 
Swedish higher education, expansion and growing complexity stand out as decisive 
features. We agree with Jacobsson et al. (2015) that growing complexity involves a 
“dual governing problem” for the state: “its internal complexity and multifunctional 
nature poses a problem for the governing of the state which, in turn, significantly 
impacts the capacity of the state to govern society” (Jacobsson et al. 2015, p. 13). 
We thus identify two ways in which the concept of complexity is informative for us 
in understanding the governing—evaluation—knowledge nexus: first, complexity as 
an “object” of governance and, second, growing complexity as an “intrinsic” aspect 
of governance. 

Often, complexity is conceptualised in line with the first interpretation, namely, 
as a kind of external predicament for state governance to act on, balance, and 
hopefully reduce. Here, complexity in the form of constant change, uncertainty, and 
unpredictability in the world is seen as a problem of state governance to address 
(Kooiman 1993; Pierre and Peters 2005; Duit and Galaz 2008; Rhodes 2011; Room 
2011; Jacobsson et al. 2015). One important notion in this literature goes back to the 
prevalence of “wicked problems”, i.e. the idea that “in the complex world of social 
planning” (Rittel and Webber 1973, p. 165), important socio-political problems 
confronting governance are inherently difficult or even unsolvable. For instance, 
problems within higher education, in terms of “quality” in knowledge production 
and teaching, may not be wicked in the same way as issues like global climate 
change or social injustice and poverty, but even so, they are complex, and they 
appear to grow even more so over time. 

Our study has dealt primarily with complexity as an inherent feature of HE gov- 
ernance and EQA, i.e. as something that evolves as the state seeks to address and 
resolve various perceived problems. According to Tainter’s definition, complexity in 
human systems is a problem-solving activity that refers to increasing differentiation 
and specialisation in structure combined with increasing integration and organisation 
of parts (Tainter 1988). Using this broad framework, our case could be summarised 
as an identification of how evaluative systems gradually evolve to encompass more 
professionals and roles; system designs that are (re)assembled and combined over 
time; and additional techniques, indicators, areas and aspects, templates, and 
information communicated through increasing numbers of channels. We have also 
highlighted the amount of work that goes into organising and enacting these 
activities undertaken within state agencies, such as the SHEA and its agency prede- 
cessors, and in HEIs. 

Looking at one aspect — information sharing — we can discern a substantial 
change over time. Until the 1990s, information about evaluations was distributed to 
a small number of recipients: the evaluated HEI, the Ministry of Education, the 
advisory committee, the government, the parliament, and libraries. Today, the 
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SHEA distributes results and other information through their website, press releases, 
newsletters, conferences, documents, and various social media platforms, such as 
Twitter, to inform a broad range of perceived stakeholders. Moreover, changes in 
the ways information is actually used as a problem-solving tool produce increasing 
complexity. Until the 1980s, evaluations primarily served the purposes of the state, 
and the assessed HEIs were seen as free to use information from evaluations at their 
own discretion (Gréjer 2004). Eventually, in the early 1990s, the idea was introduced 
that HEIs must actually use and adhere to what was presented in the evaluations. 
Follow-ups were introduced to assure compliance, and with time, economic and 
other sanctions were introduced along with stronger demands for comparability in 
the form of the idea that HEI quality ought to be comparable on the basis of 
information from national evaluations. As a result, these developments called for 
more standardised evaluations to assure that such comparability was valid, which, 
in turn, produced new forms of work as the evaluations themselves became the 
objects of examination and critique. Today, as we have shown in previous chapters, 
information processing has evolved, and the communication load has also become 
more of an internal phenomenon within HEIs. Development of IQA systems and 
external demands on HEIs in terms of, for instance, production of self-evaluations 
in new process/divisional organisations leads to more interfaces and unavoidable 
friction loss in communication between various actors, institutions, or parts of 
organisations (Warzecha 2017). Another striking example is the way in which 
information about IQA must now percolate within HEIs in ways that reveal certain 
feedback loops that constitute evidence of quality to external observers. Thus, 
whereas information about evaluation in HEIs was initially a matter for the state 
bureaucracy, it has become something that ought to be spread to and used by all 
divisions, units, and employees in each institution. In this way, our data indicate that 
EQA developments appear to contribute to more general patterns of increased 
information load and management in higher education. 

For Tainter (1988), complexity is a neutral term — that is, neither good nor bad. It 
has the potential to be a productive and functional problem-solving tool but only as 
far as available resources allow: Complexity requires sufficient energy flows for 
maintenance, and more complex systems require more energy than simpler ones. 
Tainter’s (1988) rather pessimistic conclusion from studying the collapse of 
civilisations and systems of agriculture and energy is that all living systems 
eventually reach a point of diminishing returns, i.e. a situation in which complexity 
grows too costly. On a smaller scale, systems of innovation and knowledge 
production are subject to the same evolutionary dynamics (Strumsky et al. 2010). 
This means that over time, growing research institutions have come to produce 
increasingly specialised and narrowly useful knowledge at growing costs: 


The productivity of innovation is not constant. It varies not only with incentives and knowl- 
edge capital, but also with constraints. Research problems over time grow increasingly 
esoteric and intractable. Innovation therefore grows increasingly complex, and 
correspondingly more costly. It grows more costly, moreover, not merely in absolute terms, 
but relatively as well: In the shares of national resources that it requires. Most importantly, 
as innovation grows complex and costly, it reaches diminishing returns. Higher and higher 
expenditures produce fewer and fewer innovations per unit of investment. (Strumsky et al. 
2010, p. 497) 
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Interestingly for us, and as shown in the chapter “National Evaluation Systems”, 
EQA came to be perceived as a means to address the problem of expansion and 
increasing complexity within higher education as elite institutions grew to mass 
universities in the 1960s. Such expansion involved new and growing HEIs, new 
research and teaching subjects/programmes, recruitment of new teaching staff, and 
new groups of students, which resulted in the perceived problems of efficient 
managing and steering. In addition, higher education has continuously been ascribed 
new tasks and obligations to meet the needs of a society and a working life 
characterised by similar dynamics in terms of increasing complexity. Over time, 
HEIs have continued to expand, and national EQA systems have developed 
correspondingly to support and audit these developments. In times of temporary 
economic recession, cost cutting became a dominant argument for the need for 
(national and external) evaluation. As “quality” increasingly became established as 
an umbrella term for ideal production of goods and services, evaluation remained a 
key solution to perceived problems. 

The irony of the situation is that important higher education reforms in 1977, 
1993, and 2011 related to decentralisation, freedom, and autonomy were attempts to 
solve problems of expansion and increasing complexity, but the complexities they 
intended to combat actually increased as they were enacted. EQA systems involving 
the audit of results or institutional reviews are based on similar ideas and are aligned 
to overall principles of governance through goals or objectives that have produced 
increased complexity rather than simplification. Thus, Tainter’s (1988) notion of the 
evolutionary dynamics of complexity is tangential to developments in Sweden and 
elsewhere under the label New Public Management. Over the last few decades, 
decentralisation and autonomy were introduced to combat escalating bureaucratic 
expansion and inefficiency, but this required an increased complexity of and in 
measures to ensure and enforce state control of local actors (Walsh 1995; Power 
1999). Thus, Geyer and Rihani’s (2010) general observation on public policy is 
valid for Swedish higher education policy as well: 


Once again, attempts at creating greater flexibility and variety in the administration and 
outputs of public policy were undermined by an overriding desire for central control and 
oversight. (Geyer and Rihani 2010, pp. 23-24) 


In our studies of the recent history of national EQA, we have identified processes of 
oscillation, layering, and sedimentation in evaluative systems. The question is 
whether we can expect continuing expansion of national EQA systems in which one 
system is implemented only to be replaced by another some years later. To critically 
discuss the prospects of alternative futures, resource issues are imperative: for 
example, can EQA work be conceptualised as an equation involving balance 
between quality and return on investment? For obvious reasons, there are a number 
of difficulties attached to such intellectual diversion. Starting with quality, it is 
notoriously difficult to assess whether HEIs are actually qualitatively “better” today 
than 10, 20, or 50 years ago. Moreover, the role of EQA as a driver of improvement 
in this respect is equally intricate. Looking at the economic part of the equation and 
return on investment, this component is equally intangible and tends to evade 
empirical scrutiny. We tried to empirically document the actual cost related to EQA 
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in our project. However, this turned out to be a difficult undertaking because EQA 
work is not limited to practices and actors whose work is displayed transparently in 
accounts (cf. Alkin and Ruskus 1984; Dahler-Larsen 2012; Forsell and Ivarsson 
Westerberg 2014). On the contrary, it increasingly disperses into networks, 
institutions, and people who are often not paid to do it. According to Dahler-Larsen 
(2012, p. 184), these activities “incur costs for which no one is held accountable”. 
Overall, the economic dimension of EQA remains obscure, and as long as such 
evaluative practices appear functional and legitimate, scholars will have a more 
difficult time advancing critical discussions on potential problems related to 
diminishing returns. 

A comparison with another system facing similar challenges in terms of declin- 
ing return on investment may be illustrative in this respect: Within the energy sys- 
tem, the energy return on investment (EROI) ratio can explain how much energy is 
required to deliver new energy (cf. Hall et al. 2014). However, when it comes to the 
evaluation—governance—knowledge nexus, there is no applicable model even close 
to the EROI ratio. Needless to say, it would be a nearly impossible task to provide 
valid knowledge to inform the “values” in the above imagined equation. Even so, 
the “quality return on EQA investment” and the “efficiency” of continuous expansion 
of EQA escape examination and critical scrutiny. We know from decades of 
organisational studies that this kind of managerial enterprise might provide 
legitimacy rather than improve actual performance, but we would very much 
welcome such a debate, as “evaluation gluttony” (see the chapter “Enacting a 
National Reform Interval in Times of Uncertainty: Evaluation Gluttony Among the 
Willing”) comes at a (yet largely unknown) price. As we have discussed in the book, 
such costs may extend from narrow economic values to profound human concerns 
in terms of how we view and choose to undertake education and science. 
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Christina Segerholm 


Abstract This appendix serves the purpose of informing readers about the design, the 
overall theoretical approach and methodology, and the different sub-studies, methods 
and materials in the research project that is reported in the book: Governing by evalu- 
ation in higher education in Sweden, 2013-2018. Two particular issues concerning 
our research process are also included: the redirection of one sub-study due to unfore- 
seen policy developments, and the problem of denied access. Both, we believe, may 
be concerns for researchers in the social sciences in general, that can open possibilities 
or restrict the knowledge generation process so vital for democratic societies. 


This book is the result of the comprehensive research project Governing by evaluation 
in higher education in Sweden, 2013-2018. As we reasoned in the chapter “Governing 
by Evaluation: Setting the Scene”, some of the findings were made possible because 
of the particular design of the project and its sub-studies. Therefore, a description of 
the general theoretical approach, methodology, design, and the different sub-studies 
that were the basis for the material we generated is provided.! We also touch upon the 
problem of access that we encountered in our efforts to come as close as possible to 
the actual evaluation and quality assurance processes we were interested in studying 
as a way to better understand contemporary education governing in higher education. 


'The appendix is a revision and update of the research proposal to the Swedish Research Council 
in spring 2012 (project 721-2012-5424). Unforeseen political decisions made us redirect some of 
our work to follow the events as they evolved. 
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Project Approach 


The project started out in line with the explanation-oriented (or theory-directed) 
evaluation approach (Franke-Wikberg and Lundgren 1980). This approach was 
further developed by Segerholm (2003) in order to enable critical studies of 
evaluations or of evaluation and quality assurance (EQA) systems. The latter 
approach was used to structure the project and its sub-studies. With this approach, 
several factors are related to each other in order to better understand and explain the 
consequences of EQA systems: (a) the context, intentions, and aims of EQA 
systems, (b) how they are carried out/the processes, and (c) their outcomes — results 
as expressed in reports and decisions and as experienced by representatives of the 
evaluators and evaluated. Issues of evaluation influence, during EQA processes and 
their outcomes, need to be included in such critical studies. Examples include the 
use of reports and decisions and enactment of decisions by different actors 
(Segerholm 2003). The approach also requires the development of a theoretical 
framework to help understand both processes and outcomes. This is typically done 
in an evolving process in relation to the empirical material, as was also the case in 
our project. In this book, such a stance is notable in our exploration of the relation 
between governing, evaluation, and knowledge, that is, the nexus, which is 
elaborated in the chapters “Governing by Evaluation: Setting the Scene” and 
“Evaluation Machinery, Qualocrats, and the Seemingly Inevitable Problem of 
Expansion”. The various theoretical resources used in the different chapters in the 
book are also evidence of our theoretically eclectic approach. Even so, the overall 
conceptual frame concerning the nexus is more or less visible in the chapters. 


Methodology, Design, and Data 


Methodologically, Stake’s case study approach (Stake 1995, 2006) was used since 
its logic fits well with the aims and theoretical approach of Franke-Wikberg and 
Lundgren (1980) and Segerholm (2003). It allowed us to view Sweden as a case, 
with emphasis on the latest national EQA system (the 2016 system). The case 
methodology also permitted us to carry out sub-studies and use several different 
methods for data collection and production, analysis, and reporting. A strategy of 
progressive focusing was helpful (Parlett and Hamilton 1972, p. 18; Stake 1995, 
p. 9), ensuring adaption of the line of inquiry to the shifting policy contexts, such as 
an unexpected period with the rare situation of a lack of a political decision on a 
coming national EQA system — what we call a ‘reform interval’ (see the chapter 
“Enacting a National Reform Interval in Times of Uncertainty: Evaluation Gluttony 
Among the Willing”). Progressive focusing implies flexibility to pursue issues that 
are not identified in advance. The design opted to cover a—c in the project approach 
described above, in particular concerning the 2016 national EQA system. The 
following sub-studies were carried out to achieve this: 
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Sub-study 1: Mapping the European policy context of evaluation in higher educa- 
tion and its intersection with the Swedish policy context 


Research questions: What is the European policy on evaluation and quality assur- 
ance in higher education? How does it enter Swedish policy-making spaces? To 
what extent and through what channels do European ideas on EQA in higher 
education shape or influence Swedish practices? The study included: 


(a) 


(b) 


A review of a selection of policy literature by the European Association for 
Quality Assurance in Higher Education (ENQA), official texts, web-based 
information, brochures, and letters. Also included was an analysis of the 
ENQA policy of EQA in higher education in Europe and between Europe 
and the national context of Sweden, connecting it to literature on policy 
activities through which policy is disseminated, brokered, translated, and 
interpreted to fit national/local contexts. 

A study of policy channels and influences at the intersection of European 
and national policy brokering and mediation based on interviews with ten 
national “policy brokers’ responsible for the policy contacts between Sweden 
and Europe (particularly ENQA). 


Sub-study 2: Mapping the Swedish policy context, intentions, and aims of the 2016 
national EQA system, as well as key actors’ notions of quality 

Research questions: What are the characteristics of the 2016 EQA system’s context, 
and to what extent does it diverge or converge with European ideas? What were 
the intentions and aims of the system? What different notions of quality in higher 
education are visible in the historical and present policy contexts? What forms of 
evidence do these notions of quality require? The study included: 


(a) 


(b 


wm 


(c) 


A review of the policy context and the national EQA systems in higher edu- 
cation in Sweden from the significant reforms of 1993-2018, exploring the 
main changes and the relation between European ideas on evaluation of 
higher education and Swedish ideas. 

A review of policy literature on the particular processes leading to the highly 
criticised 2011-2014 national EQA system, and to the present 2016 system, 
including the intentions, aims, and key actors, — that is, the politics of 
EQA. Interviews with key policy actors (so-called policy brokers) on the 
development of the quality evaluation system including one representative of 
the Association for Swedish Higher Education Institutions’(ASHED) task 
force for quality issues, five officers at the two national agencies responsible 
for EQA in higher education during the period under study, one representative 
from the Ministry of Education, and one student union representative, 
mapping the different standpoints and notions of quality in higher education. 
An interview study with 33 out of 35 vice-chancellors at the higher educa- 
tion institutions (HEI) listed at the national agency in 2014 as central policy 
actors in ASHEI on their notions of quality in higher education. 
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Sub-study 3: Mapping a reform interval, higher education institutions’ and the 
national agency’s preparations for a new national EQA system 

As policy developments unfolded, the study turned out to be an unexpected oppor- 
tunity to follow the process when a new national EQA system was in the making. 
Initially we applied for funds to study the 2011-2014 system, which was termi- 
nated by the time we got the research grant. At this point, we decided to redirect 
one sub-study to follow the process of the development of the 2016 national 
EQA system from an HEI perspective, in parallel with the national agency’s (the 
Swedish Higher Education Authority, SHEA’s) perspective. 

Research questions: How do HEIs prepare for a new national EQA system that is 
not yet decided? How does the responsible national agency prepare and navigate 
during such a reform interval? What is the governing potential in such a reform 
interval? 


(a) Four HEIs of various sizes, locations, histories, and previous evaluation 
judgements were studied through ten interviews with central quality 
assurance managers, faculty representatives, and teachers at the HEIs. These 
were a bit unevenly distributed depending on the size of the individual 
HEI. Documents from the HEIs, such as internal policy documents, vice- 
chancellors’ blogs, and earlier evaluation reports, were collected and 
analysed. 

(b) Interviews with four officers at the national agency, the SHEA, covering 
questions about how they worked with and prepared others to work with the 
2016 national EQA system. 


Sub-study 4: Mapping the quality evaluation regime and evaluator practice, 1.e. the 
pilot of so-called institutional reviews of the HEIs’ internal quality assurance 
(IQA) systems 

Research questions: What are the characteristics of the national EQA system’s and 
the institutional reviews pilot’s designs, and of the evaluators (SHEA officers and 
external assessment panels), and what are their notions of quality in higher 
education? How is the pilot negotiated with HEIs? How is the pilot carried out, 
what knowledge constitutes evidence, and what is the basis for judgements and 
decisions? How is the pilot used for policy learning? 


(a) A study of the design of the present 2016 national EQA system using official 
texts and internal materials from the SHEA and interviews with eight SHEA 
staff was conducted. 

(b) A study of the background, training, experience and notions of quality in 
higher education of staff responsible for the EQA system at the SHEA and 
the external experts/academic professionals (peers) was conducted. This 
included staff members’ networks, their claims to expertise (the basis for 
their judgements), the selection of external assessor panels, and their modes 
of operation (how the evaluations are planned and carried out, what is exam- 
ined, against what criteria, for how long, with what evidence). Here, we 
planned to collect information through interviews with assessors and obser- 
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vations of their work processes, such as site visits, meetings, and introduc- 
tory meetings with HEI actors in the so-called institutional review pilot. 
However, observations were denied (see further below), so we relied on 
interviews on two to three occasions throughout the institutional review pilot 
process with the SHEA project leaders for two of the HEIs in the pilot (the 
Eagle and the Falcon) and on interviews at three occasions with the chairs of 
the external assessor panels for the same two HEIs. Additional interviews 
were also conducted with one other member of the external assessor panel. 
A set of reference interviews with a SHEA project leader, and chair of the 
assessor panel, and another member of the panel for a third HEI (the Hawke) 
were also conducted. A staff person at the SHEA responsible for the training 
of the assessors supplemented this study. In order to get a holistic under- 
standing of the institutional reviews of the Eagle and Falcon, sub-study 4 was 
carried out in parallel to sub-study 5 and targeted the same HEIs. 


Sub-study 5: Mapping higher education practice of how to deal with the institu- 
tional review pilot of the HEIs’ internal quality assurance systems 

Research questions: How does the 2016 national EQA system (the pilot) enter par- 
ticular HEIs? How do key HEI actors experience and react to the pilot and the 
2016 system? How are they involved in evaluation events? To what extent do 
these processes shape their work and their views on higher education and quality 
in higher education? 


(a) 


(b) 


At the HEI level: We conducted case studies of reception and handling at the 
Eagle and the Falcon, the ‘enactment’ of the pilot as part of the national 
EQA system, its policy and practice, and its effects and consequences at the 
HEI level. This was accomplished through interviews with key actors 
responsible for quality issues at the selected HEIs and other actors taking 
part as representatives of different groups at the HEIs, such as vice- 
chancellors, programme directors, teachers, and students. At the Falcon, we 
interviewed eight persons and at the Eagle ten persons, some of them at more 
than one occasion. Local policy documents concerning the delegation of 
power, management structures, plans and descriptions of internal quality 
assurance systems, and schedules for and distribution of internal quality 
assurance related HEI activities in relation to the pilot were collected and 
incorporated in the study. As in sub-study 4, we had planned to follow these 
processes by observations of the site visits being part of the review/evaluation 
process, but that was turned down due to denial of access by the SHEA (see 
further below). 

At the teaching staff level: four telephone interviews took place with ran- 
domly selected teachers at each HEI (the Eagle and the Falcon, a total of 
eight) in which questions about their knowledge about the ongoing pilot 
were asked. We inquired if and how teaching staff at the assessed HEIs had 
noticed or experienced that their HEI had been included in the institutional 
reviews pilot. 
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Sub-study 6: Review and synthesis 


Many of the results from our project are reported in this book. Several interviews 
formed the basis for more than one study since the same informants were central 
to a number of issues and processes we studied. In synthesising and writing the 
book, we strived to integrate different types of information into a chronology, 
wrestling with the governing-evaluation-knowledge nexus, and doing justice to 
what we learned throughout the project. There are still avenues and materials that 
are not fully explored, but we believe that this project with its comprehensive 
scope concerning one case — the Swedish — and its special design, shed light on 
and may be used as a comparative basis for studies of other national/state/ 
regional EQA systems and their part in governing higher education. 


Summary of interviews* 


Vice-chancellors 33 
Policy brokers 10 
SHEA staff 5 +5 SHEA project leaders for Falcon, Eagle and 
Hawke institutional reviews pilot 
HEIs: 
Hercules 1 
Orion 4 
Pegasus 2 
Virgo 3 
Falcon 14 
Eagle 14 


External assessors 10 including assessors from Falcon, Eagle and Hawke 
institutional reviews pilot 


‘Note that the number of interviews listed here and the number of interviewees reported in the sub- 
studies are not the same. The reason for this is that some of the interviews were used in more than 
one sub-study (e.g. the SHEA project leaders and vice-chancellors) and some interviewees were 
interviewed more than once 


Most of our documentary material and interviews are in Swedish, meaning that 
all citations from texts and quotations from interviews are translated into English by 
us. We have noted in several chapters the importance of translation and are of course 
aware of the difficulties in capturing and transmitting meaning in these interpretation 
and translation processes. 


The Problem of Access 


As indicated in the above description, we had planned to observe assessment and 
judgement practise on the one hand and enactment processes at the HEIs on the 
other hand, in the same reviews/evaluation processes. In our contact with the 
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evaluation department at the SHEA, we requested access to observe the site visits 
that the external assessment panels and the SHEA project leaders carried out at the 
HEIs in the institutional review pilot. In our request, we underscored that we would 
not select review processes in which the HEIs we worked in ourselves were assessed. 
We also asked for access to the electronic systems for the external panels’ internal 
communications as well as the HEI’s uploaded material (self-evaluation reports, 
etc.). Both our requests for access to review process and materials were however 
denied. The reason was: 


When exercise of public authority is at hand, serious demands are made on administration 
in line with central requirements for the rule of law. It is important that the higher education 
institutions trust that the reviews are not influenced by irrelevant concerns. To allow 
researchers employed at some of the higher education institutions reviewed by the SHEA to 
follow the review process may risk that the administration of the review is questioned. Yet 
another circumstance to take into account is that the working material that is developed 
during the review process would be public documents when you get access to it, since these 
documents would be considered public when they come in to other authorities, Mid Sweden 
University and Umea University.? This would jeopardise or risk inhibiting the SHEA’s 
review work. (SHEA staff, email 18 March 2016) 


In the same communication, we were offered the opportunity to interview the SHEA 
staff we found necessary to get information from. We have indeed experienced good 
access to SHEA staff, and they have been willing to be interviewed and share their 
experiences and perspectives. This has of curse been very valuable to us. Still, we 
discussed this reply from the SHEA and contacted a university lawyer in order to 
find out the legal grounds for the SHEA position and answer. The information from 
the lawyer was that the SHEA could not prevent us from participating in the HEI 
site visits if the HEIs allowed us to be there. However, we decided not to proceed, 
as explicit consent from SHEA was not obtained. In December 2017, when the 
whole process of the institutional review pilot was finished and the decisions had 
been formally issued by the SHEA, we again approached the agency and asked for 
permission to access the electronic platform used by the external assessment panels, 
this time in retrospect after the formal decision, but were again denied access. 

From our previous experiences with empirical research on school inspection car- 
ried out from 2010 to 2013, we knew that observations of on-site process activities 
add important information. In this light, and having been permitted to access such 
activities in previous projects, we regret that we were not allowed to observe actions, 
interactions, gestures, mimics, and verbal communication this time. We wanted to 
accompany assessors to their HEI site visits and observe internal SHEA meetings 
and the deliberations in external assessors’ meetings. This would have added addi- 
tional and valuable dimensions to our data and analysis. As Travers put it, we 
wanted: 


*Public HEIs are national authorities in Sweden and as such subject to the principle of public 
access. The principle decrees that all documents in paper or electronic format that are kept at, 
incoming to, or developed by an authority should be open to any individual for reading. Exemptions 
are documents that are classified or restricted or directed as personal communication to a public 
officer. Public access is laid down in the Freedom of the Press Act, SFS (1949:105, Chap. 2), and 
in the Public Access to Information and Secrecy Act, SFS (2009:400). 
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to observe what actually happens when inspectors deliberate about the performance of an 
institution, or professionals review issues about quality, or managers discuss concerns about 
the performance of professionals. It is by spending time in what the American sociologist 
Erving Goffman (1959) called these ‘backstage’ settings that one can gain most insight into 
what people understand by the term ‘quality’. (Travers 2007, p. 2) 


We also think that it is important in a democratic society to be able to study the 
practices in national authorities to gain insight into how policy and governing come 
about in education (and other policy areas). We still hope this will be possible in the 
future. 
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